NuGet does a lot of neat things like resolve dependencies (actually downloading the additional packages needed and the right versions). Makes it really easy if you want to change to a different version...just modify packages.config.
I really loved RubyGems in RoR, which I think influenced NuGet. One thing you could do in RubyGems was freeze all your included gems (and any manual configuration) so that you could check them in to source control. In fact, I think the /packages directory in Solution root stores the dlls NuGet downloads.
EDIT: superblowdart beat me to /packages