SecretSoftware, maybe you're thinking of the hybrid harddrives?  ReadyBoost has no affect on when data is written to the hard drive.

The hybrid harddrives are intended to save power as well as improving performance, but ReadyBoost is purely about performance.

To answer the original question, I've had my USB key plugged in permanently (with 2.5GB for ReadyBoost) so I can't really judge how big a difference it makes, but what I do know is that my 2 year old laptop is running faster now than it ever did under XP.