DISCLAIMER: I'm a staunch anti silverlight-for-the-web guy. So my opinion is 180 degrees opposite to most here. Feel free to ignore my post
First, nice to meet a SL guy out there, its getting rarer to meet one. Second, my understanding is that true 3D ain't going to happen from your perspective. That's because you want, as i understand it, a Microsoft-approved technology. In 3D-land it would only be DirectX, and that is only on windows/xbox so no x-platformness for you. If you want true x-platform (i realized you were sold silverlight as x-platform) that would only be OpenGL/WebGL.
EDIT: It occurred to me that you might find a c# library to do the same within Silverlight.