I learned SL after spending six months developing Flex, so I think I have a good feel for both from the viewpoint of a developer. Ultimately it's true what people assume--it's more natural for a .net developer to gravitate towards SL, and java programmers to tend towards Flex. However, as my SL skills increased I came to more fully appreciate how incredible SL is as a development technology.
IMO the learning curve is a little harder than MS portrays, but the architecture is very natural once you get used to it. Two or three full-time weeks and you're not cussing any more. But I could say the same about learning .net when it first came out.
I especially love the fact that you can take almost any control and re-skin it to suit your purposes. If the control was written well, you can re style it very easily without a lot of pain.
In terms of developer experience, it's probably not fair for me to compare since I was new to Eclipse, and had many years' experience with VS. Both have been surprisingly unstable (crashing, etc.), but VS was definitely more predictable. It didn't do things that I couldn't explain, or require a project to be "cleaned" to start working again. Blend does sometimes require me to close and restart to get around what appears to be internal project caching. VS2008 does crash a lot with SL projects, (Blend occasionaly crashes too). After spending six months with each environment; there's no way I would ever go back. VS enabled me to be far more productive than Flex/Eclipse.
Two big keys to being productive these days are being able to deal with webservices and xml. I found SL/VS to make consuming my own webservices almost trivial. Easy and 100% rock solid. When using a webreference (generated proxy class) you get strongly-typed variables and built-in support for 2-way binding (via ObservableCollection). In six months of development I have never had one single problem or limitation in this area. Now, if I wanted to avoid webreferences and interrogate the returned xml from somebody else's service, I found it easier in Flex. Since my original project was written in Flex, I tried this approach at first when porting to SL and it was difficult. Since I controlled the services and didn't need to worry about the services changing without me knowing (and recompiling my SL project) it was an easy choice. But, I wonder how difficult it will be the first time I need to use somebody's returned xml without using a precompiled proxy class. As for XML itself, I definitely found it MUCH easier to use XML in SL than in Flex. As I recall, Flex used an approach similar to .net's linq whereby the 'xpath-like' instructions are compiled into the app that made it difficult to slice into documents based on variables (e.g. MyXMLElement.Descendants(MyNameVariable).FirstOrDefault()). I remember that I overcame this in Flex, but it was much more natural (for me anyway) in SL.
One thing that might not matter to an end-user, but matters a lot to me, is compilation time. I run my app (with 20k+ lines of C# and xaml) scores of times each day, with the debugger attached. SL/VS beats Flex/Eclipse hands down in this area. I'm not talking about milliseconds; I'm talking about real time that adds up over a work week.
As for runtime speed and performance in the browser, I didn't really have any complaints with either. Both seem acceptably fast to me, and run consistently in either browser (firefox or ie). Some 3rd party SL components run slowly in VS with the debugger, but that's been the only complaint I've ever had. From the user's perspective I think both technologies work great. I can't speak to download sizes, because I'm not at work right now, and my SL project has grown to many times the size of the original Flex project anyway. For the record, my medium size SL3 app runs perfectly in Chrome, even though it's not officially supported. I also can't speak to anything related to video or sound, as I don't do that kind of work.
My biggest complaint so far has been than Blend is rather slow, and that VS2008 is painfully slow to open xaml files. (Like 45 seconds to open a 1000 line .xaml file). I'm hoping that VS2010 will improve this situation.
I like the fact that SL is evolving fast. Developers want right-click support, printing support, mouse wheel support, clipboard access, etc.; and MS is listening. I haven't had the luxury of trying out SL4 beta, but I'm looking forward to it. This gives me comfort that I'm backing the right horse; that I'm not stuck with some platform that may or may not make it in the marketplace.
One big shortcoming for both environments is 3rd party components. I think component companies (at least the few I've worked with) are a little behind the curve. This is probably more true of SL, since it's changing so fast. Not to say there aren't quality components to be had--we're using a number of 3rd party components with nominal results.
My advice for any new SL developer would be this: make all your components (ui) properly skinnable with default control templates from the start. Resist the tempation to emit controls from your C# code or implement other stuff that the next guy (or gal) can't extend by editing a copy of the control's template. And learn to use Blend--watch the MS videos, and spend time with it.
I used to like winforms and GDI stuff with .the 2.0 framework, but this is light years ahead of that. You can craft a very rich UI with little effort AND have it be extensible (and run over the internet, of course...). And don't even get me started about asp.net--if you've cursed while doing ajax/dhtml stuff this is going to be like going to heaven for you.
This may be one of the coolest things Microsoft has done yet!