@itsnotabug: I'm running Fedora instead of Ubuntu (officially support) so I can't say how it works on that.
Yes, video drivers required a bit of work because I used pre-built proprietary driver packages that conflicts with some other video drivers (because packager wanted so) - lines 61-67. Open source drivers works out-of-the-box without tweaks. I installed needed NVidia files without pre-built package. Additional to that I made a small tweak that checks what video card is in use and it chooses a video card driver according to that - like you predicted
I'm going to make my own packages in the future and use kickstart that builds whole system (bootable and installable ISO-file) with latest system/drivers/applications/etc. with one command.