I just run a VM off of Windows Azure. Actually Windows Azure is pretty nice. It's not necessarily the cheapest, but you can be sure that you won't have accidentally given your credit card details to some Nigerian Prince, and if you find you don't need your VM, you can decommit it and avoid getting charged. You'll also find that Microsoft, being as it is a major corporation, is perhaps slightly more likely than some teenager in his mom's basement at actually honouring the 99.99% uptime claim on the website.
You'll also benefit from the fact that Microsoft has spared no expense in getting bandwidth and adding DDOS protection to the core of Windows Azure. Microsoft aren't taking chances with their big customers here, and you'll inherit that protection for free.
There's also a 30 or 90 day trial or something if I remember correctly. You need to give your credit card details to MS so they can be sure you're not jumping from trial to trial or hosting illegal stuffs, but that's about it.
Also, if you find after a while that you're only running a web and or SQL server, there are cheaper plans that are not VM-based as well on Windows Azure, where they manage all of the cruft like Windows Updates on your behalf.
Oh - and in the unlikely event that you find that your server isn't big enough, you can beef it up on the same VM, or elastically scale over multiple VMs either via the management console or via Powershell scripts, and you'll have access to the extra resources almost immediately. No need to speak with an adviser or wait a couple of hours.
So my advice to you if you do go Azure, is first of all decide if you actually need a full VM with RDP. If you do, get the smallest VM and scale up from there as and when you need it. But think carefully. Directly hosting your website and SQL content in Azure is cheaper and less hassle if you don't actually need the more exotic apps that only a VM gives you running in the cloud.
Also - get Server 2012 if you can. The interface is a little bit horrendous (metro and server really don't mix well) but Server 2012 is more secure and faster than 2k8-R2. And also because if you're choosing a less secure server to put on the Internet because you don't like the desktop shell GUI, then someone needs to lock you out of the server room.
If you don't go for Azure. At least go for another big company like Amazon. Seriously. That $2 a month cheaper isn't worth the trouble fighting with the bank when your credit card details end up on some russian hacker forum or fighting with the sysadmin after he only delivers you 40% uptime or a 56kps internet connection this month.