Have you checked whether it's an Ethernet MTU (MaximumTransfer Unit) issue. ?
I believe Vista has a dynamic tuning algorithm. In which case, it's possible that Vista is detecting a black hole Router, and it's reaction to that could be manifesting itself with this Server - ie the Media Server is making invalid/inappropriate assumptions
about the lower-level network transport, and eventually fails.
Just a guess. If you really want to get into it, try using WireShark or Microsoft's Network Monitor to capture the network packets as it starts to fail.