A colleague of mine had a problem and came to me. A provider of data had closed the ftp server that he used to download some files from. The files were now only available from a web site and he asked me if it was possible to download it from the web site. He sent me an email with the URL and the credentials needed to logon to the site.
I started to code the solution in VS2008, the first problem is that the site demanded forms authentication so I sent a post to the page with the data that the form needed to logon. WebException occurred! It turned out that the System.Net.WebClient refuses to connect to a site where the certificate has expired – Google code and 5 minutes later I had the solution. Now I was able to download the web page that contains the links to the different files that was available. I used a regular expression to find the anchors in the html document and I downloaded the latest file.
It took me about an hour to solve my colleague’s problem and I didn’t know much about System.Net.WebClient when I started out. Microsoft .NET made it so simple to solve this problem quickly – I love .NET!