Extreme ASP.NET Makeover: Testing - Of Tightropes and Tests

Play Extreme ASP.NET Makeover: Testing - Of Tightropes and Tests
Sign in to queue


Of Tightropes and Tests

Applying a strong suite of automated tests provides a codebase with a safety net. Developers can modify code protected by tests in relative safety. Method implementations can be changed, class hierarchies refactored, and code improved in a myriad of ways. Automated tests can be run on the code at any time to verify that existing functionality is not broken. Existing automated tests act to prevent regressions from entering the codebase. (A regression is a feature that previously worked, but does not anymore.)

Now why would a development team want to modify working code? There are many reasons:

  • A bug needs to be fixed and you want to fix it without breaking other functionality.
  • You think of a better or more efficient method of implementing a feature.
  • You want to pay down some technical debt accrued by the team.

(Technical debt is a metaphor created by Ward Cunningham to describe the gradual accumulation of friction in a codebase due to quick-and-dirty programming. As technical debt builds, the difficulty in working with a codebase increases. You can reduce this debt by refactoring the code to eliminate duplication, increase cohesion, decrease coupling, etc. Your chances of refactoring code safely increases, thereby reducing technical debt, if you have a good automated test suite acting as a safety net.)

Just like a safety net used when walking tightropes, a suite of automated tests is only as good as its coverage. Having a safety net isn't much use if you fall outside of the net’s perimeter or if the holes in the net are big enough to let you fall through. The same holds true for automated tests. If you are modifying code that isn't tested, you are walking a tightrope without a safety net.

Testing Landscape in an ASP.NET World

We will focus on developer-oriented testing, including unit, integration, and acceptance testing. These terms are often confused such that most "unit tests" are actually integration tests. Developers think that they're writing unit tests because they're using a unit testing framework. What we, as developers, call "unit testing frameworks" are in reality "testing frameworks" that can be used to write a wide variety of test types.

If most developers are actually doing integration testing, what is unit testing? Unit testing involves testing a single component, which in object-oriented terms is a class. With integration testing, we are testing a set of components working together.

There are a wide variety of testing frameworks available for the Microsoft .NET Framework. The most commonly used ones are NUnit, MbUnit, xUnit, and MSTest. All are .NET ports of JUnit, the original Java testing framework. Actually the original unit testing framework was SUnit, written by Kent Beck, for Smalltalk, but wasn't widely distributed. Beck went on to co-write JUnit with Erich Gamma and JUnit did gain widespread acceptance.

ScrewTurn Wiki already uses NUnit for unit and integration testing and we will continue to use NUnit throughout the series. The unit and integration tests apply to business, data access, workflow, and similar logic. They do not interact with the HTML and JavaScript of a Web page. These tests provide us with a safety net when working with the core application code. We are going to examine ScrewTurn Wiki's unit and integration tests in detail in a future article when we talk about refactoring the core application code. In the next few articles, we will be updating and improving ScrewTurn Wiki's HTML, CSS, and JavaScript. We need a safety net here too and so we turn our focus in this article to automated acceptance tests around the Web user interface.

Automated Acceptance Testing

Acceptance testing is black-box, end-to-end testing of a system to ensure that it meets the needs of the end user. Acceptance tests are often typified by long, verbose Word documents with step-by-step manual instructions, which are performed and verified by the QA department or end users. Here is a simple manual acceptance test:

Verify Administrative Portal Requires Admin Credentials

  1. Browse to the main page at http://server/Default.aspx.
  2. If currently logged in, click Logout.
  3. Verify that you are accessing the site as a guest. ("Welcome guest" should be displayed in the top right.)
  4. Browse to http://server/Admin.aspx.
  5. Verify that you are re-directed to http://server/Login.aspx and prompted for credentials.

A typical system will have dozens to hundreds of these tests. Manual testing is repetitive, monotonous, time-consuming, and error-prone. The immediate question that should jump to mind is, "How difficult is it to automate these acceptance tests?" As it turns out, it is a lot easier than you might otherwise think.

Introducing WatiN

WatiN is a Web application testing framework written in C#. It is an acronym for Web Application Testing in .NET and was inspired by WatiR - Web Application Testing in Ruby. To use WatiN, you simply reference WatiN.Core in your test project and start writing test code. It integrates with all of the major .NET testing frameworks. Here is some NUnit code using WatiN:

public void CanBrowseToMicrosoft() {
    using(var browser = new IE()) {
        Assert.AreEqual("Microsoft Corporation", browser.Title);

In the code above, IE is a class defined in WatiN.Core. When an instance of IE is created, an actual instance of Internet Explorer is launched. WatiN automates the browser, instructing it to go to a particular URL and retrieve the title of the resultant page. WatiN can also click on buttons and links, fill in form fields, hover over elements, and just about anything else a real user can do. Since WatiN is automating a real browser, you have real JavaScript and Asynchronous JavaScript + XML (AJAX) support. With WatiN, you can automate complex user interactions that traverse multiple pages, such as adding or editing a page on a wiki, buying an item (including a multi-step checkout) on an e-commerce site, or completing a multi-page survey on a polling site. Rather than write isolated simple tests, you can actually automate business-relevant workflows. Since you're running the acceptance tests through a testing framework, you can create a clean database and/or site with default content before every test run, which maximizes your ability to create a reproducible set of acceptance tests that can be run on demand without user interaction.

In addition to automating Internet Explorer, WatiN 2.0 (which is currently in beta) can automate Mozilla Firefox v2/v3 and has very preliminary support for Google Chrome. Eventually you will be able to do some cross-browser testing with WatiN, though not cross-platform because WatiN directly automates the browser, which must be running on Windows. We will be focusing on Internet Explorer since WatiN's support for that browser is the most mature.

Setting Up WatiN

Wouldn't it be wonderful if software just worked right out of the box every time? WatiN is a bit trickier than most. In getting WatiN running on your system, you have two hurdles to overcome. WatiN is interacting with Internet Explorer through a COM automation API. WatiN is able to hide almost all of the COM-related details from you except for one thing: the threading model. The Internet Explorer COM automation API assumes that you're executing in the single-threaded apartment (STA). This is COM-speak for only allowing one thread to interact with a component. Some test runners use a multithreaded apartment (MTA), which means any thread can access the component. For example, NUnit and MbUnit test runners both use the MTA by default. xUnit, MSTest, TestDriven.NET, and JetBrains ReSharper test runners use the STA. Note that use of a STA or MTA thread for running tests is dependent on the test runner rather than the test framework. TestDriven.NET running NUnit tests will run them on a STA thread, but NUnit GUI running NUnit tests will run them on a MTA thread. To confuse matters even more, MbUnit will override the test runner so TestDriven.NET running MbUnit tests will run them on a MTA thread! This is a long way of saying that you need to run WatiN tests on an STA thread and you better set it yourself to make sure that WatiN can successfully communicate with Internet Explorer. For NUnit, we add an application configuration file called <TestAssemblyName>.dll.config (which in our case is called WebApplication-Test.dll.config) to our test project, as shown below.

<?xml version="1.0" encoding="utf-8" ?>
    <sectionGroup name="NUnit">
      <section name="TestRunner" 
      <add key="ApartmentState" value="STA" />


On the file properties for WebApplication-Tests.dll.config, change the Copy to Output Directory to "Copy if newer."


If you are using another test framework, you can find information on the WatiN Website on how to properly configure the apartment state.

Just one more change and we'll be able to execute our WatiN tests. Internet Explorer 7 and 8 on Windows Vista or later run in Protected Mode by default. Protected Mode prevents Internet Explorer (and hence any Web content running inside it) from modifying your system. It also hinders communication between Internet Explorer and WatiN. The easiest way to disable Protected Mode for a site is to add it to your Trusted sites list. As shown below, I have added https://www.microsoft.com to the Trusted sites list to run our simple WatiN, as well as http://localhost , since we'll be using a local Webserver to host our test site.


For more detailed information on WatiN and Internet Explorer Protected Mode, see my blog post entitled "Running WatiN Tests on Vista". For more information on Internet Explorer Protected Mode, check out "Understanding and Working in Protected Mode Internet Explorer" in the MSDN Library.

With the test runner's COM apartment state set to STA and Internet Explorer Protected Mode disabled for https://www.microsoft.com, we can execute the simple WatiN test from above which browses to the Microsoft home page and verifies the page title.

Other videos from this article

· Of Tightropes and Tests

· Using WatiN

· Eliminating Repetition from Tests

· Acceptance Tests

Read the full article at https://msdn.microsoft.com/en-us/magazine/dd744751.aspx



Download this episode

The Discussion

Add Your 2 Cents