Ifeanyi Echeruo: Testing WPF - UI Fuzzing with InvokeStress | Charles

Tim Sneath

I wasn't expecting to be the first poster to this video, so I'll take this opportunity to say what an excellent job the interviewer did Smiley

You can download the tool from the following location:
http://wpf.netfx3.com/files/folders/developer/entry5155.aspx

Ifeanyi and I are standing by for any questions you might have - we're interested to see how you use this tool either for WPF or even traditional applications...

crnriman

http://channel9.msdn.com/ShowPost.aspx?PostID=226785
Will get you to the bits a tad bit quicker

Gimped
@caption goes here@

I forget which Channel9 interview it was, but someone gave a tip that when showing VS .NET onscreen it is best to use Lucida Console 14 to make it legible. That would have been handy here.

crnriman

I'll try to keep that in mind for the future.

Still you can grab the sources (5 files or so) and search for the API's mentioned </plug>

jbwebb

Hi, thanks for the interesting video.

How safe is this to leave unattended? Having tested it, I think I'd prefer to let it crunch away on a VM rather than on my main dev machine. Big Smile

Also, how difficult would it be to adapt this technique to record your (or your users') actions as you drive your application?
This would be a very fast way of building up test scenarios for later playback.

- Jonathan

crnriman

Q. How safe is this to leave unattended?

Well like I said I've been using it to internally test a couple of apps for a few months. In overnight runs it hasnt had unwanted side effects on other applications or the operating system. There is always the chance it could trigger a 'catastrophic' code path in the application being tested but thats why you would be running it to begin with.

I'd prefer running on as close to the target environment as possible but running in a VM is still an excellent option esp. since you can get freaky and still feel safe (e.g several concurrent instances on multiple accounts with varying system privilages)

Q. How difficult would it be to adapt this technique to record your (or your users') actions as you drive your application?

Recording actions is easy. Capturing enough information to reliably playback is very difficult.
The most prominent problems are
    deriving a persistent id for AutomationElements that are not explicitly labeled
    recording enough information to gate playback.
eg.
   1) Click login,
   2) wait for sign-in to complete
   3) wait for contacts list to populate
   4) Double Click on contact

Between each action is a signal that playback needs in order to issue the next action at the right time. It isnt always event driven, sometimes you may need to poll. For network based apps your environment could be wildly different with every run and the actions need to adapt.

I know there are companies that sell software that does this. I haven't played with any so I dont know what state of the art is in this field. Perhaps a knowledgable Niner can enlighten us?

Based on my experience what I would recommend writing down a scenario on paper (you did do that when you were spec'ing the app right? Wink

)
Then convert that scenario to code.

e.g
Click on button X
Select radio button Y
Select textbox Z
Type "Channel 9 Rocks"
Click Ok

try {
AutomationElement xButton = MyElementHelper.Find(uiRoot, "X", ControlType.Button);
Input.MoveToAndClick(xButton);
...
AutomationElement zTextBox = MyElementHelper.Find(uiRoot, "Z", ControlType.Text);
Input.MoveToAndClick(zTextBox);
Input.SendUnicodeString("Channel 9 Rocks", 100, 100);
....
AutomationElement okButton = MyElementHelper.Find(uiRoot, "Ok", ControlType.Button); //this is not the best way to invoke the ok button
Input.MoveToAndClick(okButton);
}
catch(MyElementNotFoundException) {
//report that the automation failed
}

Scripting this sort of automation is something I am getting more and more interested in but I have no budget (time wise) to chase this right now

jbwebb

Hi, thanks for your great response.

I ran InvokeStress against a little multi-threaded app of mine and it coped fairly well with the 'carpet bombing' input. (I've noticed that Alt+F4 seems to be the best way of aborting the driven application.)

One issue that did come to light was that my winform layout needed to be tightened up since InvokeStress set up some weird dimensions that I'd never have tried.
I'll definitely be using this in future, it's a powerful technique. Big Smile

- Jonathan

TommyCarlier
I want my scalps!

Gimped wrote:
I forget which Channel9 interview it was, but someone gave a tip that when showing VS .NET onscreen it is best to use Lucida Console 14 to make it legible. That would have been handy here.

Don Box - What goes into a great technical presentation?

m1t0s1s
Those that don't understand unix are doomed to recreate it

What about Consolas?

Although I could see lucida being good for screenshots, and consolas for getting down and dirty in the code.

Here's a very nice list of monospace coding fonts, fyi:
http://lowing.org/fonts/

Lucida console comes in at #14.

I switch between monaco (mac), dejavu sans mono (which is a rebadged version of bitstream vera, donated to the gnome project), and Microsoft's consolas.
Anonymous TT is nice too.

I have yet to find a good ascii art font for windows, though. And terminal is bleeeeeehhhhhhhhh.