Ifeanyi Echeruo: Testing WPF - UI Fuzzing with InvokeStress

I wasn't expecting to be the first poster to this video, so I'll take this opportunity to say what an excellent job the interviewer did

You can download the tool from the following location:
   http://wpf.netfx3.com/files/folders/developer/entry5155.aspx

Ifeanyi and I are standing by for any questions you might have - we're interested to see how you use this tool either for WPF or even traditional applications...

http://channel9.msdn.com/ShowPost.aspx?PostID=226785
Will get you to the bits a tad bit quicker

I forget which Channel9 interview it was, but someone gave a tip that when showing VS .NET onscreen it is best to use Lucida Console 14 to make it legible. That would have been handy here.

I'll try to keep that in mind for the future.

Still you can grab the sources (5 files or so) and search for the API's mentioned </plug>

Hi, thanks for the interesting video.

How safe is this to leave unattended? Having tested it, I think I'd prefer to let it crunch away on a VM rather than on my main dev machine.

Also, how difficult would it be to adapt this technique to record your (or your users') actions as you drive your application?
This would be a very fast way of building up test scenarios for later playback.


- Jonathan

Q. How safe is this to leave unattended? Well like I said I've been using it to internally test a couple of apps for a few months. In overnight runs it hasnt had unwanted side effects on other applications or the operating system. There is always the chance it could trigger a 'catastrophic' code path in the application being tested but thats why you would be running it to begin with. I'd prefer running on as close to the target environment as possible but running in a VM is still an excellent option esp. since you can get freaky and still feel safe (e.g several concurrent instances on multiple accounts with varying system privilages) Q. How difficult would it be to adapt this technique to record your (or your users') actions as you drive your application? Recording actions is easy. Capturing enough information to reliably playback is very difficult. The most prominent problems are     deriving a persistent id for AutomationElements that are not explicitly labeled     recording enough information to gate playback. eg.    1) Click login,     2) wait for sign-in to complete    3) wait for contacts list to populate    4) Double Click on contact Between each action is a signal that playback needs in order to issue the next action at the right time. It isnt always event driven, sometimes you may need to poll. For network based apps your environment could be wildly different with every run and the actions need to adapt. I know there are companies that sell software that does this. I haven't played with any so I dont know what state of the art is in this field. Perhaps a knowledgable Niner can enlighten us? Based on my experience what I would recommend writing down a scenario on paper (you did do that when you were spec'ing the app right? ) Then convert that scenario to code. e.g Click on button X Select radio button Y Select textbox Z Type "Channel 9 Rocks" Click Ok try {   AutomationElement xButton = MyElementHelper.Find(uiRoot, "X", ControlType.Button);   Input.MoveToAndClick(xButton);   ...   AutomationElement zTextBox = MyElementHelper.Find(uiRoot, "Z", ControlType.Text);   Input.MoveToAndClick(zTextBox);   Input.SendUnicodeString("Channel 9 Rocks", 100, 100);   ....    AutomationElement okButton = MyElementHelper.Find(uiRoot, "Ok", ControlType.Button); //this is not the best way to invoke the ok button   Input.MoveToAndClick(okButton); } catch(MyElementNotFoundException) {   //report that the automation failed } Scripting this sort of automation is something I am getting more and more interested in but I have no budget (time wise) to chase this right now

 

Hi, thanks for your great response.

I ran InvokeStress against a little multi-threaded app of mine and it coped fairly well with the 'carpet bombing' input. (I've noticed that Alt+F4 seems to be the best way of aborting the driven application.)

One issue that did come to light was that my winform layout needed to be tightened up since InvokeStress set up some weird dimensions that I'd never have tried.
I'll definitely be using this in future, it's a powerful technique.

- Jonathan

Gimped wrote:

I forget which Channel9 interview it was, but someone gave a tip that when showing VS .NET onscreen it is best to use Lucida Console 14 to make it legible. That would have been handy here.

Don Box - What goes into a great technical presentation?

What about Consolas?

Although I could see lucida being good for screenshots, and consolas for getting down and dirty in the code.

Here's a very nice list of monospace coding fonts, fyi:
http://lowing.org/fonts/

Lucida console comes in at #14.

I switch between monaco (mac), dejavu sans mono (which is a rebadged version of bitstream vera, donated to the gnome project), and Microsoft's consolas.
Anonymous TT is nice too.

I have yet to find a good ascii art font for windows, though. And terminal is bleeeeeehhhhhhhhh.

crnriman wrote:



Q. How safe is this to leave unattended? Well like I said I've been using it to internally test a couple of apps for a few months. In overnight runs it hasnt had unwanted side effects on other applications or the operating system. There is always the chance it could trigger a 'catastrophic' code path in the application being tested but thats why you would be running it to begin with. I'd prefer running on as close to the target environment as possible but running in a VM is still an excellent option esp. since you can get freaky and still feel safe (e.g several concurrent instances on multiple accounts with varying system privilages) Q. How difficult would it be to adapt this technique to record your (or your users') actions as you drive your application? Recording actions is easy. Capturing enough information to reliably playback is very difficult. The most prominent problems are     deriving a persistent id for AutomationElements that are not explicitly labeled     recording enough information to gate playback. eg.    1) Click login,     2) wait for sign-in to complete    3) wait for contacts list to populate    4) Double Click on contact Between each action is a signal that playback needs in order to issue the next action at the right time. It isnt always event driven, sometimes you may need to poll. For network based apps your environment could be wildly different with every run and the actions need to adapt. I know there are companies that sell software that does this. I haven't played with any so I dont know what state of the art is in this field. Perhaps a knowledgable Niner can enlighten us? Based on my experience what I would recommend writing down a scenario on paper (you did do that when you were spec'ing the app right? ) Then convert that scenario to code. e.g Click on button X Select radio button Y Select textbox Z Type "Channel 9 Rocks" Click Ok try {   AutomationElement xButton = MyElementHelper.Find(uiRoot, "X", ControlType.Button);   Input.MoveToAndClick(xButton);   ...   AutomationElement zTextBox = MyElementHelper.Find(uiRoot, "Z", ControlType.Text);   Input.MoveToAndClick(zTextBox);   Input.SendUnicodeString("Channel 9 Rocks", 100, 100);   ....    AutomationElement okButton = MyElementHelper.Find(uiRoot, "Ok", ControlType.Button); //this is not the best way to invoke the ok button   Input.MoveToAndClick(okButton); } catch(MyElementNotFoundException) {   //report that the automation failed } Scripting this sort of automation is something I am getting more and more interested in but I have no budget (time wise) to chase this right now

 

Hi,

I found this is a pretty useful tool to test our UI.
I have some doubt regarding adding our own scenarios / test cases.

1) Do i need to import some additional dlls / packages ?Channel9.Samples.Test.Stability"
2) Do I need to add the provided classes into some solution & rebuild the same ?
Plz let me know the steps in details

Thanks
Debashis

1) Do i need to import some additional dlls / packages ?Channel9.Samples.Test.Stability"

2) Do I need to add the provided classes into some solution & rebuild the same ?

You dont need to do either of those.
The tests run in an application separate from your products application.

In other words all you have to do is

Build my application
Run your application
Run my application with command line arguments telling it how to find your running application.