"You can easily re-create the problem by creating any new web project (.net3.5), then copy/paste client code from CTP Cache portal and poof. You get the error. Switch to .net4 and you're rolling".
So after getting no where with that problem and no replies to my question about 3.5 support for the cache client I decided to try and upgrade to .net 4.0 in hopes I could finally get moving onto azure.
After upgrading all my references to their 4.0 versions I ran into a new problem which appears to be a total roadblock. The problem is with the ReportViewer control. It apparently needs to make use of session state, and it doesn't want to use the AppFabricCacheSessionStoreProvider so I'm back to going nowhere again.