Rob Short (and kernel team) - Going deep inside Windows Vista's kernel architecture

The Discussion

• 

  First to reply.
  
  Looking forward to this one, hooray for your blog scoble.
  
  And where is the next singularity vid....
  

• 

  Happy holidays for the geeks indeed!

• 

  Great! Will like see/hear more from Rob and his team in the future.

• 

  Saptak wrote:

  Great! Will like see/hear more from Rob and his team in the future.

  
  
  We'll be talking to some of the folks Rob mentioned (the engineers who are making this stuff happen, plus more with Richard Ward on security in Windows...)
  
  Also, stay tuned for a discussion with a couple Shell architects.
  
  It's always a pleasure to listen to and learn from the big thinkers behind our technologies.
  
  C

• 

  KevinB wrote:

  
  
  And where is the next singularity vid....
  

  
  
  You mean this one?

• 

  I've enjoyed most of the Channel 9 Videos. But this one was especially cool.
  
  It would be really interesting to find out if someone at microsoft is looking at the problem of http://channel9.msdn.com/ctpmadness/ and also that software has to be installed or uninstalled in a specific order or it leaves the system in a bad state. For example I had a hard time getting CSharp Express 2005 Beta 2 off my system. I know it should not be installed on a production machine, yada yada, it's a beta, yada yada. But why must it effect, or should I be so bold as say infect, my system like that?
  
  

• 

  Good lighting in this video. I hope every video has the same level of lighting as much as possible. I hate dark videos where you can barely see the person. Channel 9 needs to invest in some semi-pro lighting equipment. I know this goes against the philosophy of not being intimidating for the interviewee, but it will go a long way in making videos more clear.

• 

  Slides19 wrote:

  Good lighting in this video. I hope every video has the same level of lighting as much as possible. I hate dark videos where you can barely see the person. Channel 9 needs to invest in some semi-pro lighting equipment. I know this goes against the philosophy of not being intimidating for the interviewee, but it will go a long way in making videos more clear.

  
  
  Sorry, there's a lot of reasons why I will never use lights or professional microphones. It comes down to access and having a real conversation. I'm getting conversations no one else has gotten. Why? Because of access and because I don't make people feel uncomfortable.
  
  Have you been in a TV studio with lights or have to put on a microphone? I have. It makes you very aware that you're being filmed. It makes you behave differently than you might if you just were meeting for lunch with some friends. It makes you less comfortable. (Not to mention that having to mike up four people like in this video is just impossible unless you have an audio crew along and then we're not doing a spontaneous one-hour meeting, we're doing something different.
  
  I also need to be able to walk in, setup, and start shooting. Why? Because that makes it far more possible to get access to these types of people (and to conference rooms -- you did notice that at the end of this one we got kicked out of the conference room, right? These things are booked around the clock by teams). The more I have to setup equipment, the more time I'll need (I can't walk across campus with much heavy equipment, for instance -- it's already hard enough to get around campus with a 15lb tripod).
  
  I try to get good lighting, but am stuck with what's there. Sorry about that, but my first priority is the conversation.

• 

  Fair enough, it's your call and I can understand not wanting to carry equipment around all day long. Though I would suggest using well lit rooms like this one as much as possible when you gather folks for a group interview.

  First time posting on Ch9, been watching your videos for a while and I really enjoy them. This has been a real PR coup for MS, even if very subtle.

• 

  Where the guy was talking about them having the tech to find dependencies for binaries I thought they already had that. Thats what Depends(Start->Run->Depends (May need Visual Studio installed, I cannot remember where its actually located...))  does.
  

• 

  IRenderable wrote:

  Where the guy was talking about them having the tech to find dependencies for binaries I thought they already had that. Thats what Depends(Start->Run->Depends (May need Visual Studio installed, I cannot remember where its actually located...))  does.
  

  
  
  That doesn't quite do the job in Windows, from what I'm told. What they need to know are the TEAMS who are affected by those dependencies.

• 

  Slides19 wrote:

  Fair enough, it's your call and I can understand not wanting to carry equipment around all day long. Though I would suggest using well lit rooms like this one as much as possible when you gather folks for a group interview.

  First time posting on Ch9, been watching your videos for a while and I really enjoy them. This has been a real PR coup for MS, even if very subtle.

  
  
  Unfortunately I have very little control over where shooting will happen. That's all scheduled by the people I'm shooting. I don't think I've shot in the same place twice.

• 

  It's a tad more complicated than that, IRenderable...

  C

• 

  I would like to see more information about what is happening to make creating shell extensions and namespace extensions easier.  It seems to be black magic to get them coded and debugged. 

  I have made a couple and even though I have plenty of ideas about what I would like to do, it is a pain in the neck to make them. 

  I only bring this up because one of the guys mentioned that there are some people looking into this af far as extensibility is concerned.

• 

  I really enjoyed this video, it's nice to know that "real" developers are still necessary today, even for "boring" stuff like Windows kernel.

• 

  Enjoyed the movie a lot.
  
  Anyone that has had a problem with the registry knows it can be a beast but overall the registry is a great system and with a transactional filesystem for it to sit on I'm sure we're going to see improved MTBF. As for cleaning up the registry I'd say thats an excellent idea - I hope the registry wasn't missed in the LUA guidelines; natrually that should facilitate a large portion of the cleanup.
  
  From the singularity video and this one I'm left with one question: Has the WHQL coming up with more strict criteria for kernel space drivers, specifically video cards? It seems video card and motherboard drivers are going to be the only drivers that will never be able to touch user space due to latency concerns.
  
  PS: I look forward to hearing from Anders Vinberg about aoughtness and isness.

• 

  Excellent video, Happy Hollydays to everyone!

• 

  deedubb wrote:

  From the singularity video and this one I'm left with one question: Has the WHQL coming up with more strict criteria for kernel space drivers, specifically video cards? It seems video card and motherboard drivers are going to be the only drivers that will never be able to touch user space due to latency concerns.

  
  
  VDDM display drivers run in user mode in Vista. There is a kernel mode miniport as well, but the main driver runs in user mode. Some of the things they talked about RE: recovery from driver crashes applies here. In Vista, if a display driver faults, Windows can reload the driver or swap it with a different driver without having to reboot. Driver upgrades can also be handled w/o a reboot. During the upgrade, Windows unloads the current IHV driver, reverts to the standard VGA driver, then loads in the new IHV driver.

• 

  Probably the December CTP or Intel/Realtek drivers still need some work in the less glitching audio area since just watching this video (from HDD) had many moments where the audio went into continous glitch for second or two and video halted and resumed after few seconds. In background I was installing the Beta Client, but XP would not have had any glitches at this kind of little stress test. The hardware is intel 915G with Realtek ac97 audio. Maybe the eye candy is too much for the onboard Intel graphics solution and thus glitches. The audio is also crackling in the video, but restarting it removed the crackling.

• 

  Charles wrote:

  KevinB wrote: And where is the next singularity vid....

  
  
  You mean this one?

  
  
  At the end of that one, you said you hoped to revist the team to discuss some more about Singularity and get a demo of the OS.
  
  Whilst I find most of the cideos on chan9 very interesting, the singulairty one, and most of the going deep series for that matter, are particularily enjoyable.
  

• 

  KevinB,
  
  The Singularity folks will be visited again in 4-6 months. They're rather busy researchers and I don't want to wear out our welcome
  
  C

• 

  - "I don't regret the registry was developed; It's unfortunate that the registry was overused"
  - "overused is the wrong word for the registry. The real problem with the regsitry is that we never actually defined the set of guidelines and schema for how people should use the registry - and if we had done that, we wouldn't be in this mess today."

• 

  Architecture, must master it sometime

• 

  bariswheel wrote:

  - "I don't regret the registry was developed; It's unfortunate that the registry was overused"
  - "overused is the wrong word for the registry. The real problem with the regsitry is that we never actually defined the set of guidelines and schema for how people should use the registry - and if we had done that, we wouldn't be in this mess today."

  
  Do ask why the registry was overused.
  As I know, Windows Users(not professional developers, or power users) dislike touch the registry at first, but the registry is designed that 1)Windows depend on it too much, and 2)'terrorists' are easy to write a piece of code or javasicrpt(through IE) to affect the use of Windows, at final, the Windows Users have to google 'the register' for get relative knowledge about the register.
  The same thing is , you guys write an application(windows) with a Button(the registry), and you tell your customers DONOT click the Button(the registry), because it may damage the application(windows), that's ridiculous.
  
  I personal think the register is more like DOS(or win98...), it's a good idea(or good product) at its time, but now, I have to say it's out of date.
  
  

• 

  Download link of the video doesnot work.
  

• 

  I agree /w Rich Neves towards the end of the tape. The future wont be the "one-size-fits-all"-OS/Kernel. Listening to these "Biggies" gives me a glimpse of how painfull it must be to go through the code and "componentize" the system. It must be a "pain-in-the-youknowwhat" re-designing/refining this extremly complicated and complex code-snippets. I can only guess...
  
  A lot of things that were discussed (virtualization,usermode-device-driver-framework, fragmentation, memory_management) reminded me of the discussions I follow with the linux-kernel-newsgroup (- no flaming intended).

  The "Do you guys wish (that) the Registry would have never been developed?" was sure a fun question which led to some serious answers. Thats what I like.

  Unfortunately I had some difficulty to understand what Richard Ward was saying. I guess I was not the only one.

• 

  I plan on talking to Richard Ward (solo) regarding his work in more detail given its importance and implications.
  
  Stay tuned,
  C

• 

  Nice vid.  Love that stuff. Too bad you could not get Cutler to join the fun.
  
  Seems the direction is a bit like Singularity already with SIPs in user mode and maybe SIPs in kernel mode and the line is beween upper and lower sips and between peer SIPs.  Then use message passing (channels) between all sips.  Now that would truly be some seperate black boxes with effectively *one input queue as the single "API" for each SIP and the Contract becomes the *messages.  The apis become private implementation details inside each sip.  Not sure if parsing messages becomes the perf killer or if that can be (or has been) sorted out.  That would be interesting.
  
  What was the Obfuscator in the kernel thing? Have a vid scheduled on that?  Thanks again!
  --
  wjs

• 

  Qucik Poll:
  At the End of 2006 What OS Will you be Running?
  
  http://www.dpolls.com/PollPage.aspx?PollID=57

• 

  n4cer wrote:

  
  VDDM display drivers run in user mode in Vista. There is a kernel mode miniport as well, but the main driver runs in user mode.

  
  
  I was reviewing the topic and found http://channel9.msdn.com/ShowPost.aspx?PostID=62083 which seems to say that actually the video card driver can't be moved out of the kernel address space yet... I'd like some clarification but maybe a forum thread would be a better place to persue it.

• 

  I have a urgent request for the Vista team. Please, please, please include a checkbox near the start of the setup process that will allow users to opt-out of being protected from themselves. Yes, I understand that user ignorance has been at the core of the black eye Microsoft has gotten over security. Yet, not all users are idiots. If you were only allowed to eat when my computers caught a virus or malware, you’d be dust by now! It has only happened twice, and I’m 49.

  What I’m really saying, of course, is that Vista–and its setup–must allow for differing levels of user competency. I love XP, but I can’t begin to tell you how many times it has hobbled me as I tried to complete a legitimate task. For my own usage, I need less security interference, not more.

• 

  I also have a quick poll question.
  
   Does Windows Vista go too far in security and impede usability?
  
  http://www.dPolls.com/PollPage.aspx?PollID=108

• 

  Prepare yourselves boys, you've been Slashdotted.
  

• 

  IRenderable wrote:

  Where the guy was talking about them having the tech to find dependencies for binaries I thought they already had that. Thats what Depends(Start->Run->Depends (May need Visual Studio installed, I cannot remember where its actually located...))  does.
  

  
  
  Depends doesn't work - consider COM and LoadLibrary.  These also introduce dependencies but can't be tracked by depends.  Also consider RPC interfaces - an RPC client depends on its server.
  
  These guys have some amazing static analysis tools, when they caught one of my dependencies, I was amazed - I hadn't even realized they would find that dependency.

• 

  Great video - I too enjoyed hearing what these guys are thinking about and working on. I would really like to hear more about how they are trying to manage state and state dependencies. One example mentioned was the case of two apps using the same registry setting. They may not be dependent by calling each other, but they are certainly dependent through that bit of state. I don't think the video really gave much information about that aspect of their work and would like to hear more. Also, as far as static analysis goes, are these techniques going to work with more "dynamic" languages such as Python or Ruby? Given the recent beta release of "IronPython", this may be more than an academic question. Again, thanks for the great video! The setting, lighting and sound were excellent and all participants seemed very comfortable!

• 

  That was an interesting show.
  
  How about, another interview with Steve Lacey and the Flight Sim team? The first one was awesome, and now they have just announced a few details about the next version (FSX).
  
  Jim.

• 

  Great Interview!
  
  So who trains MSFT Product Managers? Do they have an internal goup or use an specific training partner?
  
  

• 

  Alot of this went over my head.  Specificly, what was he talking about when he said they "had mapped the functional dependencies between binarys"? 
  
  ..and why is that such a good thing?
  

• 

  I think videos like this one with Rob Short are *GREAT* because they provide context for understanding product and how it got that way and how people think it should evolve.  Learning about product in a contextual vacuum is, for me at least, an unrewarding exercise.

• 

  Comment removed at user's request.

• 

  Comment removed at user's request.

• 

  It really seems like windows is broken. From the pervasiveness of the registry to the lack of modularity and isolation - the kernel seems to be a jumbled mess. On a plus note, it is great to see that they are starting the discipline of component isolation. The question is how deepley coupled these components are? Would it just make more sense to build an entire new system? Or does it make more sense to iteratively partition the system along componet lines? Or maybe Singularity is the savior. Great video!
  
   

• 

  I could not play the videos. Some times It is saying network error.

• 

  clever, interesting and insightful stuff, thank you.

• 

  "... how reliable, scalable, secure, etc Windows Vista actually will be"
  
  LOL, no to my and many others' experiences