Jeffrey Van Gogh: Testing Rx with Pex

Play Jeffrey Van Gogh: Testing Rx with Pex
Sign in to queue


Ever wonder how the Reactive Extensions get tested? Jeffrey Van Gogh gives a glimpse at how they do it. The Rx developers have been using Pex and writing parameterized unit tests. In this video, we look at Enumerable.Zip and how we can use Pex to help testing it.

Jeffrey also explains how they use Pex in their build process to regenerate the entire unit test suite on each build!

The Research in Software Engineering team (RiSE) coordinates Microsoft's research in Software Engineering in Redmond, USA.



Download this episode

The Discussion

  • User profile image

    The end kinda made me jump. I turned the volume up to hear him and the end clip went "BAM!".


    Cool stuff. I like the idea of having pex generate new tests at check-in. My continuous integration server is going to be getting some changes.



  • User profile image

    Sorry about the end... we experimented with recording the movie sound through the phone which was not a great success.

  • User profile image

    cool stuff Smiley

    recording with a phone is probably ok but you might want to do some audio mixing afterwards.. also [for me atleast] the sound is kinda off sync, but again, that could be sorted in post


    also, i'd be very interested in exactly how the Rx team integrated pex with tfs Smiley

  • User profile image

    We didn't integrate Pex with Tfs.


    In our team we use a checkin system popular inside Microsoft that does verification before checkin. Source changes get packaged up, send to a server. The server kicks off a build with the code changes, starts up a bunch of vms, installs the product and runs test on it. Only if everything passes the code is checked in by the server.


    In this system adding pex is very easy as you can write test scripts in any (scripting) language you want as long as there is a commandline to execute. Not sure how much work it would be to get it going in TFS...

  • User profile image

    oh, i see Smiley

    i got confused because one of the major new features of tfs 2010 is what is called gated check in (i belive) that does pretty much what you said, it makes a tfs shelve set and builds that and run a set of MsTest tests and if they pass, the shelve set gets checked in.. i just assumed that was what you ment Smiley


    but if you're not using tfs, what are you using? svn+hooks? source safe? or is it an internal source control system?

  • User profile image

    an internal source control system from before TFS (we're all old school Microsofties who have the old system's shortcuts in our fingers Smiley)

  • User profile image

    As Jeffrey mentionned, Pex comes with a command line that allows to re-generate the test suite. In fact, the Pex Visual Studio addin launches the same command line when running Pex.


    The command line comes with an option to specify to Pex that it should fail the run whenever a new *failing* test case is generated. By using this flag, you can ensure that your test suite stays 'Pex clean'.

  • User profile image

    i see Smiley

    it would be an interesting exxperiment to try and get pex into a tfs 2010 build definition.. it should be possible..


    @peli do you know what the differences are between pex and mstest output wise? i assume both parses the output of the commandline util as text basically..


    very interesting indeed.. i will have to do some noodeling over this Smiley

  • User profile image

    Pex and mstest produce XML files but they are radically different. What's your idea?

  • User profile image

    Hmmmmmm.....I believe that what has been implemented here is a step backwards from TDD.

    In traditional TDD you write out your contract in human readable test names with human readable failure messages. When I go to read the unit tests for a class I can just collapse to definitions and just read the name of the tests. When the tests run and fail I can tell why. I imagine Zip2() is not a very helpful test name. Also I imagine maintaining the one large coupled Pex meta test method would be clumsy and easy to accidentally remove functionality.


    I would have thought a bunch of test methods like:


    [TestMethod] public void Zip_method_with_uneven_inputs_will_have_length_of_shortest_input() {....} [TestMethod] public void Zip_method_invokes_combinator_function_with_index_equilvalent_values_from_inputs() {....} [TestMethod] public
     void Zip_method_yields_combinator_function_result() {....}



    still protects you from the implementation detail and is more explicit about the intention of the contract.


    My $0.02




Add Your 2 Cents