It depends on a bunch of variables...If you originally had low-quality PC2100 and then bought tried and true PC2700 from, say, Kingston...The differences would be more noticeable.
The size of the RAM comes into play, as well...If you have 256MB of PC2100 and then upgraded to 256MB of PC2700, you'll notice more difference than if you had a gig of PC2100 and then jumped to a gig of PC2700.
The biggest differences you're likely to see are when overclocking...A stick of PC2700 would (arguably) be more up to overclocking than a stick of PC2100 would...Especially if you further the results with RAMsinks.
EDIT: When overclocking the front side bus, I should clarify.
Now, a jump from PC2100 to PC3200 would be more noticeable.
It all comes down to how much money your willing to spend...If you have the bucks, by all means replace all your PC2100 with PC2700...but if your on a budget, you'd probably be better off adding more PC2100, especially with DDR-RAM prices rising. That is unless all of your slots are taken, in which case you would have to replace, not add RAM.