Silverlight streaming still sucks on Channel9. Maybe it's just for us brits as other people report no problems. I think you should investigate serving from somewhere else that doesn't have routing problems though - I never have any trouble with youtube, vimeo or even the smaller, less well known video sites.
And while I can't categorically say it's because the Silverlight player sucks because I don't know any other sites that use Silverlight for video, it would be nice if you could pause the player and let it buffer the whole file like you can with Quicktime and most Flash players. With the buffer running out once a minute and it taking 20 seconds to rebuffer how much the player optimistically thinks it needs to play without stopping (despite ample evidence to the contrary), Channel9 videos can't be watched without downloading them.
Oh, and there's no high-rez download for non-Windows machines, just the poxy mp4 iphone version.