If I recall correctly, Vista is supposed to be doing most of it's graphical rendering in video memory and using the GPU.
This is supposed to improve performance as GPUs are quite fast these days and it takes off a lot of load from the CPU.

(Edit: forgot the catch)

So if your video adapter is slow and lacks memory, performance ain't gonna be great. This is probably why increasing the shared memory helped.