WM_IN

Marina Polishchuk: VC++ QA - The best job in the biz

Download this episode

Download Video

Description

Meet Marina Polishchuk, a software engineer on the VC++ quality control team. Her mission: Outsmart the compiler with clever programs that confuse and or break it. Fun stuff. Here, we learn about what she does, why she does it and how she got here. Enjoy!

Tag:

C++

Embed

Format

Available formats for this video:

Actual format may change based on video formats available and browser capability.

    The Discussion

    • ChrisP
      I've been using the Microsoft VC product since v1.0 on Windows 3.1.  I'm always appreciative of the QA that goes into the compiler especially.  It drifted from the C++ standard several times but they've always brought it back in line (mostly).  I've never found any big bugs in the compiler, just the IDE environment (which has a different QA team).
    • Minh
      I'd never thought about what it would take to QA a compiler before. QA in my previous companies haven't been at all technical. What they were good at is trying weird stuff on the UI to try to break stuff.

      So, to QA a particular feature in the compiler, you must know that feature really well, right? Or knowing it well is actually a hinderance -- because you sorta gotta buy into that usage pattern.
    • Skriker V1.0
      And you get apid to do this!!! Id love this job Smiley
    • Massif
      My old boss excelled at breaking stuff after it had passed all necessary tests and QA.

      I think his record was breaking something without even touching it (it was an IR remote thing, and he just put his hand between transmitter and receiver for a second - the whole thing went titsup.)
    • eyeris
      Please give this girl her own tag. Maybe... "Marina"?
    • j0217995
      That was really interesting and I don't code. Great interview team Smiley
    • Human​Compiler
      eyeris wrote:
      Please give this girl her own tag. Maybe... "Marina"?


      In C9v4, there are tags for those who appear in videos so you can see everything someone has been in.
    • JChung2006
      Minh wrote:
      So, to QA a particular feature in the compiler, you must know that feature really well, right?

      White box testing.
      Minh wrote:
      Or knowing it well is actually a hinderance -- because you sorta gotta buy into that usage pattern.

      Black box testing.
    • Marina-
      Minh wrote:
      I'd never thought about what it would take to QA a compiler before. QA in my previous companies haven't been at all technical. What they were good at is trying weird stuff on the UI to try to break stuff.

      So, to QA a particular feature in the compiler, you must know that feature really well, right? Or knowing it well is actually a hinderance -- because you sorta gotta buy into that usage pattern.



      In the case of C++, solid testing of scenarios enabled by a new feature definitely requires knowing it well, in my opinion (even tests for simple scenarios can always break as the compiler evolves, which is one motivation to have tests at all levels of complexity in place).  If you're referring to the implementation itself, I think knowing the specifics of implementation details can sometimes  be helpful in coming up with a nice breaking test case, but may not be the most productive way to write thorough tests on the whole.

      In my experience, the ability to break the compiler usually comes from using a feature in conjunction with other elements of C++ (including itself)  in an atypical way (e.g.: testing A<5> vs. A<B<7>::value> where B<T> inherits from A<T>...). 
    • Minh
      Marina- wrote:
      
      In my experience, the ability to break the compiler usually comes from using a feature in conjunction with other elements of C++ (including itself)  in an atypical way (e.g.: testing A<5> vs. A<B<7>::value> where B<T> inherits from A<T>...). 


      Sounds interesting. Testing seems to be very systematic, then. So you build a tree of all the possible thing you can do in a particular feature, then, say, run through each scenario.

      So, C++ is a fairly static language, even with templates. The compiler does most of the work, but what happen when you're QA'ing a more dynamic language? I guess you just replace the compiler with the run-time... hehe... "just"...

      I wonder if a language like Ruby would be an order of magnitude harder to QA than C++...
    • DannyJ

      Real interesting - sounds intense, gives me a better understanding of all the complexities going on every time I hit Ctrl-Shift-B Smiley

      Keep up the good work Wink

    • Flea++

      Marina,

      I am astonished at the fact that you really never did anything with computers until your freshman year in college and then end up working on the V C++ team at Microsoft! That is very impressive!

      Did you go to the University of Washington or somewhere else?  What were you favorite computer science classes and why?

      Thanks,

      Flea++

    • DannyJ
      Yeah I'm all for the name tag thing. Smiley
    • Marina-

      Flea++ wrote:
      
      Did you go to the University of Washington or somewhere else?

      No, I went to schools in the Midwest (of comparable size).  

      Flea++ wrote:
      
       What were you favorite computer science classes and why?

      For me, enjoyment of specific courses was highly dependent on the quality of the professor, and the lack of professor's attachment to giving busy work.  My favorite general topics in CS were: Theory of Computation, Cryptography, Databases, and all of the Compiler/Programming Languages courses.

    • Flea++
      Yes---I would agree.
    • wleara

      Content removed. Please do not post comments like this.
      C

    • Merlyn

      Got here from link off Sutter's blog.  I notice this was posted quite a while ago, but I think I'll comment, anyhow.

      Watching the video definately gets me thinking about how fun it would be to work as an SDET on the VS team.  The interviewer is a little screwy, but you seem to fend off his strange questions pretty well, Marina =)  Of course, I'm sure it's difficult to be in either position (interviewer/interviewee), so good job to both of you.

    • deoxriboz

      As the interviewer mentioned lots of times:D, it's really cool to somehow force the compiler. Really loved your job

    • Poonaka
      Really interesting insight into how they test the compiler.  I wish the interviewer would have done a better job asking questions as it would have been nice to learn more.
    • umer_a

      The job title, "Software Development Engineer in Test" is in fact as ambigous as overloading a function on the basis of return type, which can confuse most compilers. In formal QA,

      SDE sitting in test?
      or
      SDE operating in test?
      or
      ..

    Comments closed

    Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.