Comment Feed for Channel 9 - Jeffrey Van Gogh: Testing Rx with Pex http://ecn.channel9.msdn.com/o9/previewImages/100/512695_100x75.jpg Channel 9 - Jeffrey Van Gogh: Testing Rx with Pex Ever wonder how the Reactive Extensions get tested? Jeffrey Van Gogh gives a glimpse at how they do it. The Rx developers have been using Pex and writing parameterized unit tests. In this video, we look at Enumerable.Zip and how we can use Pex to help testing it.Jeffrey also explains how they use Pex in their build process to regenerate the entire unit test suite on each build! The Research in Software Engineering team (RiSE) coordinates Microsoft's research in Software Engineering in Redmond, USA. en Tue, 21 May 2013 11:51:09 GMT Tue, 21 May 2013 11:51:09 GMT Rev9 Re: Jeffrey Van Gogh: Testing Rx with Pex The end kinda made me jump. I turned the volume up to hear him and the end clip went "BAM!".

 

Cool stuff. I like the idea of having pex generate new tests at check-in. My continuous integration server is going to be getting some changes.

 

Thanks

posted by DuNuNuBatman

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633965124350000000 Tue, 15 Dec 2009 22:20:35 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633965124350000000 DuNuNuBatman Re: Jeffrey Van Gogh: Testing Rx with Pex Sorry about the end... we experimented with recording the movie sound through the phone which was not a great success.

posted by peli

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633965159160000000 Tue, 15 Dec 2009 23:18:36 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633965159160000000 peli Re: Jeffrey Van Gogh: Testing Rx with Pex cool stuff Smiley

recording with a phone is probably ok but you might want to do some audio mixing afterwards.. also [for me atleast] the sound is kinda off sync, but again, that could be sorted in post

 

also, i'd be very interested in exactly how the Rx team integrated pex with tfs Smiley

posted by aL_

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633965626700000000 Wed, 16 Dec 2009 12:17:50 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633965626700000000 aL_ Re: Jeffrey Van Gogh: Testing Rx with Pex We didn't integrate Pex with Tfs.

 

In our team we use a checkin system popular inside Microsoft that does verification before checkin. Source changes get packaged up, send to a server. The server kicks off a build with the code changes, starts up a bunch of vms, installs the product and runs test on it. Only if everything passes the code is checked in by the server.

 

In this system adding pex is very easy as you can write test scripts in any (scripting) language you want as long as there is a commandline to execute. Not sure how much work it would be to get it going in TFS...

posted by J.Van.Gogh

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633965778550000000 Wed, 16 Dec 2009 16:30:55 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633965778550000000 J.Van.Gogh Re: Jeffrey Van Gogh: Testing Rx with Pex oh, i see Smiley

i got confused because one of the major new features of tfs 2010 is what is called gated check in (i belive) that does pretty much what you said, it makes a tfs shelve set and builds that and run a set of MsTest tests and if they pass, the shelve set gets checked in.. i just assumed that was what you ment Smiley

 

but if you're not using tfs, what are you using? svn+hooks? source safe? or is it an internal source control system?

posted by aL_

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633966000420000000 Wed, 16 Dec 2009 22:40:42 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633966000420000000 aL_ Re: Jeffrey Van Gogh: Testing Rx with Pex an internal source control system from before TFS (we're all old school Microsofties who have the old system's shortcuts in our fingers Smiley)

posted by J.Van.Gogh

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633966022180000000 Wed, 16 Dec 2009 23:16:58 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633966022180000000 J.Van.Gogh Re: Jeffrey Van Gogh: Testing Rx with Pex As Jeffrey mentionned, Pex comes with a command line that allows to re-generate the test suite. In fact, the Pex Visual Studio addin launches the same command line when running Pex.

 

The command line comes with an option to specify to Pex that it should fail the run whenever a new *failing* test case is generated. By using this flag, you can ensure that your test suite stays 'Pex clean'.

posted by peli

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633966027020000000 Wed, 16 Dec 2009 23:25:02 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633966027020000000 peli Re: Jeffrey Van Gogh: Testing Rx with Pex i see Smiley

it would be an interesting exxperiment to try and get pex into a tfs 2010 build definition.. it should be possible..

 

@peli do you know what the differences are between pex and mstest output wise? i assume both parses the output of the commandline util as text basically..

 

very interesting indeed.. i will have to do some noodeling over this Smiley

posted by aL_

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633968499680000000 Sat, 19 Dec 2009 20:06:08 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633968499680000000 aL_ Re: Jeffrey Van Gogh: Testing Rx with Pex Pex and mstest produce XML files but they are radically different. What's your idea?

posted by peli

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633968578440000000 Sat, 19 Dec 2009 22:17:24 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c633968578440000000 peli Re: Jeffrey Van Gogh: Testing Rx with Pex Hmmmmmm.....I believe that what has been implemented here is a step backwards from TDD.

In traditional TDD you write out your contract in human readable test names with human readable failure messages. When I go to read the unit tests for a class I can just collapse to definitions and just read the name of the tests. When the tests run and fail I can tell why. I imagine Zip2() is not a very helpful test name. Also I imagine maintaining the one large coupled Pex meta test method would be clumsy and easy to accidentally remove functionality.

 

I would have thought a bunch of test methods like:

 

[TestMethod] public void Zip_method_with_uneven_inputs_will_have_length_of_shortest_input() {....} [TestMethod] public void Zip_method_invokes_combinator_function_with_index_equilvalent_values_from_inputs() {....} [TestMethod] public
 void Zip_method_yields_combinator_function_result() {....}

 

 

still protects you from the implementation detail and is more explicit about the intention of the contract.

 

My $0.02

Lee

 

 

posted by LeeCampbell

]]> http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c634120344250000000 Sun, 13 Jun 2010 14:00:25 GMT http://channel9.msdn.com/Blogs/Peli/Testing-Rx-with-Pex#c634120344250000000 LeeCampbell