Anyway...if the user turns off the aero glass features, do all apps still end up being rendered as "vectors"? Would that cause backward compatability issues? I'm just curious as to what the bottlenecks really are...the security thing doesn't work for me because there will always be security issues.
Apps (non-WPF ones at least) aren't rendered as "vectors" either way. When the DWM is enabled (which is required for glass), the window's surface is a texture that is rendered on a 3D surface. When it is disabled, the apps render as they did on previous versions of Windows. So that probably has less to do with compatibility.
The biggest compatibility problems ("bottlenecks" as you put it) are poorly written apps. This includes those that use undocumented features or data structures (Raymond Chen's blog is chock full of examples), or those that choke under LUA.