Yep, you can definitely make your URLs cleaner, to remove file extensions, query strings, etc. You'll find it to be very similar to ASP.NET MVC in general concept (and even a bit in syntax), so hopefully the learning curve is smooth.
But, one thing to note is that they'll never be quite as good as pure server-side URLs because Silverlight (like Flash, AJAX, etc.) can only update the part of the URL after the hash ("#"), so you'll get
http://some/server/url#some/client/url - careful thinking can still lead to something very nice, you're just stuck with the "#" in the middle somewhere.