Ultimately, you're typing into a text editor, compiling, and running stuff. VS might be the best IDE right now, but who says that things have to be that way? The desktop is only an interpretation of the desired workflow, and its existence is driven by habit. An alternative interpretation might work just as well, if not better.
If you argue a Metro device can remote to a VM running VS, then I'd say that VM is desktop... no where to go.
No..I'm imagining a future in which VS-future doesn't look like VS-today. Maybe it's decoupled across devices or simply across mini apps and tiles in Windows. Who knows? No one knows if it's good or bad, so I'm not sure if it makes sense to fear it, except for maybe the learning curve that comes with the transition.