Pex - Automated Exploratory Testing for .NET

thats so awsome

i like the floating "we catch your bugs" note in the background too 

This is the coolest thing I've seen in a long time.

For those interested here are the blogs from Nikolai and Peli:
http://blogs.msdn.com/nikolait
http://blog.dotnetwiki.org/default.aspx

I can't wait for this to hit 1.0!

Nikolai and Peli,

Very interesting technology - looking forward to v1.0 and a 64-bit version. Speaking of 64-bit, when will you release a beta that supports that version of the framework?

As far as I am aware, 64bit version isn't planned any time soon. My suggestion - use a 32bit virtual machine.

Been using pex a little (not had time to really get into it yet), but its really impressive.. great work on this guys!

Too bad about the 64-bit version. We're more and more developers running in 64-bit environments (even in development), so it would really really nice to have access to the entire suite of tools. Running in a virtual environment when doing unit testing is probably one possible workaround, but I would definitely welcome 64-bit support.

I've subscribed to the PEX list but haven't seen any trafic yet.

Automated unittesting is worth dying for if done right. Can't really think of any code that wouldn't benefit (I suspect they even used Pex to test Pex ).

ive been playing around a lille with pex now and i just have to say.. WOW :O
its GREAT! it sounded cool in the interview and all but in reality its even more awsome!
if you havent already, you Have to check this out! the vs integration is sweeet
the last time i had this kind of experience was when i tried out wpf! unit testing is no longer a horrible bore but actually something interesting

again, great work pex team! cant wait for the next release

i do have a qustion though is there a way to get pex to try a generic method with a bunch of diffrent types? i guess i could use like a list of instances of the various types and get pex to use diffrent lists or diffrent indexes in the list (like with a static list and a int argument to the parameterized method)

but im sure there is a smarter way

aaah, awsome and i see it also takes multiple parameters too, sweet

is there also a way to have a staging method be called before each test or at the start of each test run?
by stageing method i mean a static method that sets up the environment in diffrent ways. i guess i could call it from within the parameterized test but maybe the pex team thought of something more clever here as well
 
im sorry if thats in the docs somewhere or in the ms test api thats built into vs, im sort of a noob with this kind of testing... im used to writing eeeverything myself.. that is also why pex makes me ever so happy

maybe that isnt even a smaart thing to do.. i guess im sort of trying to do some mini system tests in my unit tests.. but i figure, whats a unit anyway.. a part of a system could be a unit even if its actually a system in it self right?

oh i got another question too
is there a way to have the parameterized test pick some of the parameters from somewhere (like a collection) and try and figure the other ones out? like if i have a method that takes some arguments and a file path to like a config file and i want it to try out a few diffrent config files with various permutations of the other arguments to the method
so basically what i waant to do is databind to some of the arguments but have pex figure out the others

--
staging method: if the code that is being run in the method is relevant, you really want  to call it from the parameterized test. Otherwise, you can use [TestInitialize], [TestCleanup] methods, they are supported by Pex.
--

alright i'll try that  

--
Since you are mentioning the environment, be aware that Pex does not understand the environment (i.e. Pex cannot build constraint over the state of the file system, network, current time, etc...).
--
no i understand that that would be kind of spooky.. and awsome that is why it would be cool if one could supply a list of alternatives somehow

--
Partial databinding: interresting idea but it's not supported as is right now. Currently, create multiple copies of the test or use the 'Abstract Test Pattern'.
--
alright one could work around it by adding an int to te parameterized test, assume its within the range of a collection defined somewhere and use the int to look up the alternatives from that collection, but i guess pex wouldnt know that its supposed to use all the ints in the range..  you could also use if statements though, then pex would try to satisfy them but if there are alot of alternatives that might be cumbersome..
perhaps a dictionary with an enumeration as a key? im just letting my mind wander..


again, great work pex team

had some more fun with pex

is there a way to get a certin pexmethod to use one factory and another (or all others) to use some otherway of exploring classes?

or perhaps that factory souhld be a pex factory but just a regular static method that is called from the parameterized test?

also, if you want your custom classes to appear in the table of results and arguemnts in visual studio, should you use PexValue.AddForValidation? (for arguments that are custom classes as well?) or is there some other way to do that?

great stuff!
thanks

I've been playing with it for a while. Even though I know I shouldn't use it with production code (this is a research project), but I had to run it against real, production code to see some real results. Seems like me and my fellow team members forgot to write some very important tests

I hope to find this in a future release of Visual Studio.

I really like the concept. However I have some doubts ...

1. How would Pex fare if the system is a state machine ? How would it limit the number of constraints it would need to test out. ?
2. What about unsafe blocks in code ?
3. Also when you say bugs these are purely functional right ..

Also is there any place I can have a look at a sample test report .
I am really looking forward to a fully functional release of Pex.

Regards,
Anand

Why can't constraints be constructs that can be applied on the class and methods and automatically enforced on the caller by the framework? That is, one should be able to decorate a method with:

[Pre-Constraint: i>=1, i<=int.MaxValue, c!=null]
[Post-Constraint: i==i, c==c]

//Pre-constraint indicates value of i accepted from caller must be greater or equal to 1; and that c cannot be null.
//Post-Constraint imply that on method exit the constraints should also be respected otherwise the method cannot successfully complete - constraint exception condition

The above constraint would be enforced on calling code at design or compile time (if a known value or a variable whose value is known is being passed). The constraint on class instances being used in a method should then be passed up the chain to all callers. This would at least reduce a lot of run-time time issues since the constraints are enforced at design or compile time.

In fact the i<=int.MaxValue constraint would be superfluous if int already defines constraint on acceptable values of between MinValue and MaxValue.

---
Taiwo

This looks really interesting, but obviously can only work to a point due to undecidability constraints.  So how sophisticated does it get?  If I have in my code:

if (SeventhOrderPolynomial(x) < 0)...

will Pex actually try to solve the polynomial and put in values of x where it is larger than zero and others where it is smaller?  Does it give up at a certain order?  Linear?  Diophontine equations?  Does it use some sort of Monte Carlo algorithm to guess at values which may or may produce different branches in the code?

I'm impressed regardless.  Very nice work.