And of course, OpenGL...although it's not typically associated with "desktop apps". Then again, it would be a lot of fun to change that perception...
There are a handful of OpenGL custom-built UIs out there.
Blender has its own widget toolkit built with OpenGL. Clutter is another one (targetted at set top boxes) and various people are building some toolkits like that.