Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Jeffrey Van Gogh: Testing Rx with Pex

Download

Right click “Save as…”

Ever wonder how the Reactive Extensions get tested? Jeffrey Van Gogh gives a glimpse at how they do it. The Rx developers have been using Pex and writing parameterized unit tests. In this video, we look at Enumerable.Zip and how we can use Pex to help testing it.

Jeffrey also explains how they use Pex in their build process to regenerate the entire unit test suite on each build!

The Research in Software Engineering team (RiSE) coordinates Microsoft's research in Software Engineering in Redmond, USA.

Tags:

Follow the Discussion

  • The end kinda made me jump. I turned the volume up to hear him and the end clip went "BAM!".

     

    Cool stuff. I like the idea of having pex generate new tests at check-in. My continuous integration server is going to be getting some changes.

     

    Thanks

  • Peli de Halleuxpeli Peli

    Sorry about the end... we experimented with recording the movie sound through the phone which was not a great success.

  • Allan LindqvistaL_ Kinect ftw

    cool stuff Smiley

    recording with a phone is probably ok but you might want to do some audio mixing afterwards.. also [for me atleast] the sound is kinda off sync, but again, that could be sorted in post

     

    also, i'd be very interested in exactly how the Rx team integrated pex with tfs Smiley

  • We didn't integrate Pex with Tfs.

     

    In our team we use a checkin system popular inside Microsoft that does verification before checkin. Source changes get packaged up, send to a server. The server kicks off a build with the code changes, starts up a bunch of vms, installs the product and runs test on it. Only if everything passes the code is checked in by the server.

     

    In this system adding pex is very easy as you can write test scripts in any (scripting) language you want as long as there is a commandline to execute. Not sure how much work it would be to get it going in TFS...

  • Allan LindqvistaL_ Kinect ftw

    oh, i see Smiley

    i got confused because one of the major new features of tfs 2010 is what is called gated check in (i belive) that does pretty much what you said, it makes a tfs shelve set and builds that and run a set of MsTest tests and if they pass, the shelve set gets checked in.. i just assumed that was what you ment Smiley

     

    but if you're not using tfs, what are you using? svn+hooks? source safe? or is it an internal source control system?

  • an internal source control system from before TFS (we're all old school Microsofties who have the old system's shortcuts in our fingers Smiley)

  • Peli de Halleuxpeli Peli

    As Jeffrey mentionned, Pex comes with a command line that allows to re-generate the test suite. In fact, the Pex Visual Studio addin launches the same command line when running Pex.

     

    The command line comes with an option to specify to Pex that it should fail the run whenever a new *failing* test case is generated. By using this flag, you can ensure that your test suite stays 'Pex clean'.

  • Allan LindqvistaL_ Kinect ftw

    i see Smiley

    it would be an interesting exxperiment to try and get pex into a tfs 2010 build definition.. it should be possible..

     

    @peli do you know what the differences are between pex and mstest output wise? i assume both parses the output of the commandline util as text basically..

     

    very interesting indeed.. i will have to do some noodeling over this Smiley

  • Peli de Halleuxpeli Peli

    Pex and mstest produce XML files but they are radically different. What's your idea?

  • Hmmmmmm.....I believe that what has been implemented here is a step backwards from TDD.

    In traditional TDD you write out your contract in human readable test names with human readable failure messages. When I go to read the unit tests for a class I can just collapse to definitions and just read the name of the tests. When the tests run and fail I can tell why. I imagine Zip2() is not a very helpful test name. Also I imagine maintaining the one large coupled Pex meta test method would be clumsy and easy to accidentally remove functionality.

     

    I would have thought a bunch of test methods like:

     

    [TestMethod] public void Zip_method_with_uneven_inputs_will_have_length_of_shortest_input() {....} [TestMethod] public void Zip_method_invokes_combinator_function_with_index_equilvalent_values_from_inputs() {....} [TestMethod] public
     void Zip_method_yields_combinator_function_result() {....}

     

     

    still protects you from the implementation detail and is more explicit about the intention of the contract.

     

    My $0.02

    Lee

     

     

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.