Yes, it really works.  I have a laptop that's noticeably faster with USB drive with 1G ReadyBoost cache.   It's not an instant speedup, of course; what it helps is a system under memory pressure.  If your system  bogs down during operation because you're running a number of apps and the OS is swapping, then ReadyBoost will help.

See http://blogs.msdn.com/tomarcher/archive/2006/06/02/615199.aspx