Windows, Part IV - Dave Probert

I would definitely like to see more information about the Object Manager and how the subsystems work (e.g. how a process chooses a subsystem and what implications that has on the interfaces available to it). Also, maybe some discussion on the future of subsystems (ok, Win32 "won" -- does that mean the concept of multiple subsystems is somewhat obsolete and from now on we'll all build on top of Win32, or does it make sense to have, say, a .NET subsystem if .NET is really going to supercede Win32?).

This series is the best ever... I love it... Thanks C9!

This was the most informative series I have ever seen on the Kernel.  Being primarily an ASP.NET web developer there isn’t much discussion at this level.  I loved your Mac argument of the PC driving down Mac prices.

David Dimmer
Director of Development

Fyin, Inc.
Local:  (414) 273-0101

W:  http://www.fyin.com           
Complete Website Design and Interactive Programming     

It's my understanding that one of the architectural goals of WinFX is to bypass Win32 API completely, and have .NET 2.x talk right to the kernel.

More, More!

That was by far the best series of video I have ever seen on Channel9.  Please do more like this!

Stephen.

Charles wrote:

Glad you like this! Stay tuned, you'll see more of the kernel in the near future. Next up? IO Manager and Cache Manager. Then, dig into the Memory Manager and Object Manager. Let's not forget about File Systems, eh.

I love this stuff too which is why you'll be seeing more and more deep stuff around here.

C

Ahmen to that but can you get them to include the future work as well as the history - I was just screaming about indego at the screen.

Charlie

Charles wrote:

PerfectPhase wrote: More, More! That was by far the best series of video I have ever seen on Channel9.  Please do more like this! Stephen.

Glad you like this! Stay tuned, you'll see more of the kernel in the near future. Next up? IO Manager and Cache Manager. Then, dig into the Memory Manager and Object Manager. Let's not forget about File Systems, eh.

I love this stuff too which is why you'll be seeing more and more deep stuff around here.

C

Any GUI stuff in the future? We got a brief overview
already...

It really amazes me that something as complex as NT can be demystified to this point that even an undergrad college student with a decent OS class can understand a lot of these intricacies! 

I'm really getting the sense though, that the whole argument in favor of garbage collection is really kinda overblown.  If ref counters are used at the OS level with a good deal of success, there should presumably be a way in .NET to turn off the garbage collector and use an XYZ ref counter system instead for real-time/networking kind of stuff...which is what seems problematic when everything goes the way of .NET (even real-time programming) in Longhorn.

There really ought to be a better way to schedule than using various garbage collection algorithms (Java, .NET, etc.) which take out memory after the fact at some semi-random time in the future...maybe it's an NP-hard problem?

Pure reference counting systems have issues as well, such as if two objects take out a reference on each other then the reference count will never drop to zero, hence why you need reference tracking systems such as the garbage collector that can work out these circular dependancies.

The is a paper somewhere, by Chris Sells I think where he modifies the Rotor code to use a hybrid system of counting and tracking.  Ah, here it is http://www.sellsbrothers.com/spout/default.aspx?content=archive.htm#refCountRotor

Stephen

This is the best series I've ever seen on here, and the most timely as well. I think I'm going to make my semester project in assembly on the Windows kernel, with this as my primary source (providing that's OK with you guys). Thanks a lot!

That's awesome. I'm gonna be loading a whole bunch of videos shortly that you might like.

I'm pretty sure that HAL stands for Hardware Abstraction Layer, at least that's what the MOC (Microsoft Official Curriculum) courses say (yes, I'm an MCP )

But Hardware Adaptation Layer maybe is a better definition in these times.

BTW, I loved this video too, it actually made the whole architecture very easy to understand!

Great session. It gave me a broader insight in the working, and the issues involved, of an operating system, in particular Windows NT. I'd love to see sessions in the same vein on the upcoming Windows 'Longhorn'.

Thanks Dave for your time, your a great teacher.

I am in the third video so far... This stuff is AWESOME... Definetly stuff ALL developers should have some understanding of. This is great that it is being covered on Channel9. Just when I though you were running out of stuff to talk about you find more gems...

As everyone else says, this series (I'm new around here, so I haven't watched that much) is great. And it really reminds me why the OS course you take in college is one of the most important classes you can take.

We wrote a simple throw-away 3-4000 loc alpha kernel, and it teaches multiprogramming to the bone. It was nice to see alot of the stuff (also from my CPU class was still kicking around in my head), and to hear about "compare & swap".

And I'll be looking forward to more on this stuff. I never found the time to read (it's not light reading after all) all of my copy of "Inside Windows 2K", so maybe I'll pick some up here

However, I think the most enjoyable part, was in the first (or was it second?) part, where he explained why he went with Windows, and how his son motivated that. Alone the story of how his wife was doing flyers (or something?) in troff?? My god, the pain she must have felt ...   I mean, I think LaTeX is damn sexy, but making my mom use it?

Who is the interviewer we seem to see all the time? The one with the longhorn jacket? Is he hired to do nothing but produce these videos? Because they're getting posted awefully fast ...

That's Charles Torre. He's one of the developers on Channel 9. In other words, he actually writes code. Unlike me. I don't write code. I'm just a geek with a video camera.

The posting pace is definitely pretty fast. Hard to keep up with.

Great job channel 9! Squeeze Dave and get interviews from other kernel gurus too!!!

For years, I wondered why the heck only Windows (and I believe VMS) has an implementation of the famous WaitForMultipleObjects(Ex) functions. I now understand it is the "OB" which enables this magic.

I want to hear more about the scheduler, the "OB", the "MM" and the heap management differencies between Windows NT, 2000, XP, etc. I also want to hear more about synchronization problems and their solutions in the kernel, especially about the containers (lists, queues) - how are they syncrhonized, are they lock free, how, etc. Also about the upcoming ALPC (aka LPC++?) and what's advanced (or ++) there, etc., etc.

I'm not a driver dev (middleware mostly) so these things really intrigue me. And Dave is just such a great talker!

I want more!!! and *big* thanks for doing this Channel9!

P.S. Aargh, and I forgot a very important thing! How do you guys actually *test* the kernel? I mean how the heck can one test WFMO for example? Thanks!

Great videos guys - easily the best series to date - thanks a lot, just the right depth, although I thought I was feeling a touch of the 'bends' towards the end, will have to view using helium next time

Currently just studying a university module on concurrency and these videos really bring to life the concepts we’re learning.

How about an interview with the man himself, Dave Cutler? Having read Show Stopper it seems we might not have gotten NT without him.

Ed.

I've tried getting an interview with Dave. He told me he doesn't give interviews. I'll keep on him and Raymond Chen.

Charles wrote:

Glad you like this! Stay tuned, you'll see more of the kernel in the near future. Next up? IO Manager and Cache Manager. Then, dig into the Memory Manager and Object Manager. Let's not forget about File Systems, eh.

I love this stuff too which is why you'll be seeing more and more deep stuff around here.
C

Great news! I really enjoyed this series. Ended up watching all four videos back to back at around 2AM =P

Looking forward to seeing more.

Robert, if you don't take an interview from Dave C, "your * is grass, and I'm the lawnmower"

I was under the impression from this video that .Net will become something along the same level as the win32api.  Does that mean that .Net will be using the NTDLL function calls directly? I thought that currently it uses the win32 apis to do what it does.

Kevin

It continues to amaze me how much prior art had to be reinvented by the mini/micro world.  One hopes that these concepts weren't truly reinvented, but that existing source material was consulted.

http://v3.espacenet.com/textdes?&DB=EPODOC&IDX=EP364000&QPN=EP0364000

At the link above is a 1985 patent describing how one vendor moved some resource-intensive tasking overhead from software to an auxiliary processor to improve performance once hardware began to get cheaper.  Things like multiple waits had been routine for two decades.

It is true though that Computer Science programs at an undergrad level rarely get very deep into the mechanisms supporting these things.  They are probably more frequently encountered in vendor provided OS Internals courses.

It is great to see Microsoft making this type of background more available to customers.

Great stuff guys,

As Probert mentioned in the beginning of the series, i am amazed to know the internals of the operating system. Even though Linux is my main area of interest, should definitely give you M$ guys for building the ecosystem we( yeah include me too) thrive on. It will be good if Dave Cutler also throws light on some of the design decision and his perspective of the windows kernel.

Thanx a lot guys for the good stuff thoroughly enjoyed it

Hats Off to Probert ( I am reading your paper on OS Kernel-less OS)

-Nishanth

These videos have really been fantastic! Could any of you chaps recommend a book that goes deep into the internals of WinNT/WinXP?
Thanks in advance!
Cheers!  

Time for the almost obligatory Windows Internals link. It is by far and away the best reference material on what goes on inside the bowels of Windows.

A truly fascinating and insightful read.

Thanks Andyc,
The book looks most interesting... can't wait for the book to arrive!

Cheers!
Anand Balaji.

I've ALWAYS been into MS OS's... one reason why I went into deving - because its hard to dev an OS, manage each little thing, making sure it executes something correctly etc...

I LOVE Microsoft OS's... its such a power ful thing and not alot of people relize this. Finally, a video that satisfies my MS OS fetish.... (hehe)

After watching hours of video on this site, this gotta be the most interesting and informative video yet. When he filled the whiteboard for the n-th time with lots of NT-details I was expecting him to tell you in the end that he now had to kill you all and burn the tape (and Mr Gates himself showing up at the door with a shovel and some good boots for desert-work ). We want more!

Excellent !!! Kudos !!!

Bart

 have a basic doubt. Whether memory to memory transfer using system DMA controller is possible in windows?
If not why?
Please reply back me on onypappan@yahoo.com">sonypappan@yahoo.com or sonypappan@hotmail.com
Regards,
Sony

I've been saying for years the only feasible technical solution to the Windows Anywhere vision is to effectivly pump the LPC over port 80. It's good to see more evidence of that in this latest batch of videos.

They DID NOT put http.sys in the Win2k3 kernel for 'performance' reasons like make webpages faster. They put it there for 'performance' reasons like make an internet accessible message loop.

This is the *only* windows stack that works moving forward...

------------------------
|   managed host  |  [1]
------------------------
|   SQL OS             |  [2]
------------------------
| Kernel                 |  [3]
------------------------

[1] the managed host will talk to the SQL OS
[2] http://blogs.msdn.com/slavao/archive/2005/02/05/367816.aspx
[3] everything underneath ntdll.dll