Marina Polishchuk: VC++ QA - The best job in the biz

I've been using the Microsoft VC product since v1.0 on Windows 3.1.  I'm always appreciative of the QA that goes into the compiler especially.  It drifted from the C++ standard several times but they've always brought it back in line (mostly).  I've never found any big bugs in the compiler, just the IDE environment (which has a different QA team).

I'd never thought about what it would take to QA a compiler before. QA in my previous companies haven't been at all technical. What they were good at is trying weird stuff on the UI to try to break stuff.

So, to QA a particular feature in the compiler, you must know that feature really well, right? Or knowing it well is actually a hinderance -- because you sorta gotta buy into that usage pattern.

And you get apid to do this!!! Id love this job

My old boss excelled at breaking stuff after it had passed all necessary tests and QA.

I think his record was breaking something without even touching it (it was an IR remote thing, and he just put his hand between transmitter and receiver for a second - the whole thing went titsup.)

Please give this girl her own tag. Maybe... "Marina"?

That was really interesting and I don't code. Great interview team

Minh wrote:

So, to QA a particular feature in the compiler, you must know that feature really well, right?

White box testing.

Minh wrote:

Or knowing it well is actually a hinderance -- because you sorta gotta buy into that usage pattern.

Black box testing.

Minh wrote:

I'd never thought about what it would take to QA a compiler before. QA in my previous companies haven't been at all technical. What they were good at is trying weird stuff on the UI to try to break stuff.

So, to QA a particular feature in the compiler, you must know that feature really well, right? Or knowing it well is actually a hinderance -- because you sorta gotta buy into that usage pattern.

In the case of C++, solid testing of scenarios enabled by a new feature definitely requires knowing it well, in my opinion (even tests for simple scenarios can always break as the compiler evolves, which is one motivation to have tests at all levels of complexity in place).  If you're referring to the implementation itself, I think knowing the specifics of implementation details can sometimes  be helpful in coming up with a nice breaking test case, but may not be the most productive way to write thorough tests on the whole.

In my experience, the ability to break the compiler usually comes from using a feature in conjunction with other elements of C++ (including itself)  in an atypical way (e.g.: testing A<5> vs. A<B<7>::value> where B<T> inherits from A<T>...). 

Marina- wrote:


In my experience, the ability to break the compiler usually comes from using a feature in conjunction with other elements of C++ (including itself)  in an atypical way (e.g.: testing A<5> vs. A<B<7>::value> where B<T> inherits from A<T>...). 

Sounds interesting. Testing seems to be very systematic, then. So you build a tree of all the possible thing you can do in a particular feature, then, say, run through each scenario.

So, C++ is a fairly static language, even with templates. The compiler does most of the work, but what happen when you're QA'ing a more dynamic language? I guess you just replace the compiler with the run-time... hehe... "just"...

I wonder if a language like Ruby would be an order of magnitude harder to QA than C++...

Real interesting - sounds intense, gives me a better understanding of all the complexities going on every time I hit Ctrl-Shift-B

Keep up the good work

Marina,

I am astonished at the fact that you really never did anything with computers until your freshman year in college and then end up working on the V C++ team at Microsoft! That is very impressive!

Did you go to the University of Washington or somewhere else?  What were you favorite computer science classes and why?

Thanks,

Flea++

Yeah I'm all for the name tag thing.

Flea++ wrote:


Did you go to the University of Washington or somewhere else?

No, I went to schools in the Midwest (of comparable size).  

Flea++ wrote:


 What were you favorite computer science classes and why?

For me, enjoyment of specific courses was highly dependent on the quality of the professor, and the lack of professor's attachment to giving busy work.  My favorite general topics in CS were: Theory of Computation, Cryptography, Databases, and all of the Compiler/Programming Languages courses.

Yes---I would agree.

Content removed. Please do not post comments like this.
C

Got here from link off Sutter's blog.  I notice this was posted quite a while ago, but I think I'll comment, anyhow.

Watching the video definately gets me thinking about how fun it would be to work as an SDET on the VS team.  The interviewer is a little screwy, but you seem to fend off his strange questions pretty well, Marina =)  Of course, I'm sure it's difficult to be in either position (interviewer/interviewee), so good job to both of you.

As the interviewer mentioned lots of times:D, it's really cool to somehow force the compiler. Really loved your job

Really interesting insight into how they test the compiler.  I wish the interviewer would have done a better job asking questions as it would have been nice to learn more.

The job title, "Software Development Engineer in Test" is in fact as ambigous as overloading a function on the basis of return type, which can confuse most compilers. In formal QA,

SDE sitting in test?
or
SDE operating in test?
or
..