Comment Feed for Channel 9 - Neal Christiansen - Inside File System Filter, part II

Re: Neal Christiansen - Inside File System Filter, part II

littleguru — Mon, 24 Jan 2005 21:22:01 GMT

Nice video! I like the part where he says that sometimes in the future there could be a microkernel in the bottom and drivers above them (also written in managed code). Very cool idea.

posted by littleguru

Re: Neal Christiansen - Inside File System Filter, part II

rhm — Mon, 24 Jan 2005 21:34:59 GMT

I think Neal may have fallen for this hoax email

Photograph shows RAND Corporation's 1954 design for a home computer.

posted by rhm

Re: Neal Christiansen - Inside File System Filter, part II

Drexthepimp — Mon, 24 Jan 2005 22:39:52 GMT

I have to say this guy was a pleasure to watch. He was entertaining and candid. There was a lot of intelligence to what he was saying, and I learned a lot. Dana was an interesting addition. Both guys knew their stuff, and this is exactly what channel 9 should strive to get more of. Who else would talk about the windows kernel with any authority?

I want to see some interviews with the inventor of c#, or the PE file designer. I recently went through winHex for file recovery, and there is a wealth of stuff to learn. If the linux kernell is open source, this is not going to be a major breach:)

Good job lads

Drex

posted by Drexthepimp

Re: Neal Christiansen - Inside File System Filter, part II

Senkwe Chanda — Mon, 24 Jan 2005 22:51:55 GMT

Good stuff. This was a great series. Guess I'm Neal Christansen fan now. Funny how noone had the heart to tell him that he'd been hoaxed though

posted by Senkwe Chanda

Re: Neal Christiansen - Inside File System Filter, part II

Charles — Mon, 24 Jan 2005 22:59:49 GMT

Drexthepimp wrote:

I want to see some interviews with the inventor of c#

Check out the Anders Hejlsberg videos.

Charles

posted by Charles

Re: Neal Christiansen - Inside File System Filter, part II

dnrfan — Tue, 25 Jan 2005 00:59:44 GMT

No Audio when viewed externally in WMP.

posted by dnrfan

Re: Neal Christiansen - Inside File System Filter, part II

Charles — Tue, 25 Jan 2005 02:09:24 GMT

dnrfan wrote:

No Audio when viewed externally in WMP.

I'm not experiencing this problem. Which version of Windows are you running? Which version of WMP? Could be a problem with the stream due to network conditions? The video file is fine.

Charles

posted by Charles

Re: Neal Christiansen - Inside File System Filter, part II

androidi — Tue, 25 Jan 2005 07:54:17 GMT

Charles: This goes bit offtopic, but what kind of servers host these sites (blogs.msdn.com channel9).

Should world go 90% or more underwater tomorrow - would the survivors a) have a backup located in place highly unlikely to go underwater b) if they attempted to rescue MS websites and Longhorn, would that be possible without flooding the place where the information is stored (the datacenter would need to be accessible underwater without difficulty). c) Perhaps there should be a floating satellite link and satellites which can store all the data currently on the ground.

Am I serious? Well good question, hard to answer. Recently there has been so many floods, even HERE where never to my knowledge has been such floods, that thinking these kind of possibilities cannot be entirely avoided. And I love watching those serious looking documentaries that tell about disasters and show nice graphs to support them - I have no time to go do background research of the subjects unfortunately.

My wild imagination brings me the thought of a city designed to survive the waves and then float. Like houses seen on some recent news footage. Internet would still need to work underwater - but that it does already!

Oh and while this is nothing compared to what you see on TV, here is some pics I managed to find..
http://pajalle.com/misc/tulva/

edit: Oh and needless to say, I loved these videos, but you knew that already Right?!

posted by androidi

Re: Neal Christiansen - Inside File System Filter, part II

dnrfan — Tue, 25 Jan 2005 08:02:34 GMT

Charles wrote:

Which version of Windows are you running? Which version of WMP? Could be a problem with the stream due to network conditions? The video file is fine. Charles

WMP : 10.00.00.36.46
Windows Version: Windows XP, 5.01.2600
( Version 5.1 (Build 2600xpsp_sp2_rtm.040803-2158 : Service Pack 2 ) )

If I double click the small window, I have audio, but when I click the link that makes it play in a separate WMP instance, there is no audio.

Up to you. I don't care, just reporting to you, thats all.

posted by dnrfan

Re: Neal Christiansen - Inside File System Filter, part II

Dana — Tue, 25 Jan 2005 15:41:26 GMT

I'm glad you liked it. It was fun to do. Thanks to Robert for coming over and filming a bunch of us kernel geeks.

- Dana Epp

posted by Dana

Re: Neal Christiansen - Inside File System Filter, part II

rhm — Tue, 25 Jan 2005 16:05:15 GMT

I'd like to see a video of kernel debugging in action using that windbg that was mentioned. You know, just to see how hairy it really is in there.

posted by rhm

Re: Neal Christiansen - Inside File System Filter, part II

eddwo — Wed, 26 Jan 2005 22:19:16 GMT

He mentioned the idea of having lots of different virtual machines sitting on a microkernel, isn't that just what Xen is trying to do?
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/
This is different to VMWare and Virtual PC because it does not require a fully featured OS to host the virtual machines.

From the sounds of it Xen has already progressed quite a lot and they even made some modifications to Windows XP to make it more suitable for running as a child OS in this way.

Virtual Server is more like a sooped up VirtualPC than a new architechture isn't it?

Truely fascinating video, I am very interested in the kernel side, at least from a theoretical perspective. I'm afraid my mind has been too polluted by working with VB for me to get up to that level of sophistication as a programmer.

Theres far to little knowledge about the design of NT out there, which leads to the rediculous predictions that MS will soon dump NT in favour of building a Windows compatibility layer on top of Linux.
http://www.wired.com/wired/archive/13.02/microsoft.html

Its actually more like the other way around, as Linux has evolved over time to include more portability, abstractions and extensibility layers, features that NT had designed in from the beginning. Though perhaps those features made NT too heavyweight when it was first introduced.

posted by eddwo

Re: Neal Christiansen - Inside File System Filter, part II

Charles — Wed, 26 Jan 2005 22:31:38 GMT

eddwo wrote:

Theres far to little knowledge about the design of NT out there

Not for long!

Charles

posted by Charles

Re: Neal Christiansen - Inside File System Filter, part II

Taskerr — Wed, 26 Jan 2005 23:27:10 GMT

Good video, its a credit. Or is it?

Now C9 are in the in the movie business and doing it so well its good manners to also name the camera-man, sound-man and the interviewer as well as the interviewee's in a fixed location piece like this one.

It would also be nice to see a list of published material such as whitepapers, books etc. It could afterall supplement someones income.

I must admit to feeling a little jealous of the kernel designers and maintainers in that, and I don't know for sure, but I guess the scope of what they do has remained manageable and somewhat stable over the years.

Whilst for the rest of us in the application layer - well, change is the name of the game and the scope sometimes feels infinite and overwhelming.

The application layer has seen an explosive growth in support technologies for application design. I just wish that the same was available for OS design. I'd love to see thousands of prospective OS being churned out, each one relevant to the moment and not constrained by commercial or backward compatibilty hangups. Maybe this is so and I'm just out of touch.

Perhaps Neil [et al], given their collective experience, should write a book(s) along the lines of Donald Knuth and the MIX language but instead of a lanuage define a model OS from first principles.

posted by Taskerr

Re: Neal Christiansen - Inside File System Filter, part II

prog_dotnet — Thu, 27 Jan 2005 00:19:39 GMT

I would recomend Operating Systems: Internals and Design Principles, 4/E and/or 5/E

http://vig.prenhall.com/catalog/academic/product/1%2C4096%2C0130319996%2C00.html

There is an extract from the book ( in .pdf format) that contains all the information covering windows.

You can find it on this page:
http://williamstallings.com/OS4e.html

or go directly to the document:

ftp://shell.shore.net/members/w/s/ws/Support/OS/W2K.pdf

posted by prog_dotnet

Re: Neal Christiansen - Inside File System Filter, part II

andyandy — Sun, 30 Jan 2005 12:43:49 GMT

rhm wrote:

"I'd like to see a video of kernel debugging in action using that windbg that was mentioned. You know, just to see how hairy it really is in there."

There has been a bit of that in earlier videos. Check out the video when Jason Zander gives a tour of the CLR team. Somewhere they stop with some guy who is debugging a GC error with WinDBG. Also check out Jason Zander's blog where he earlier wrote about using SOS(Son of Strike) in WinDBG to debug .NET applications there.

posted by andyandy

Re: Neal Christiansen - Inside File System Filter, part II

PerfectPhase — Tue, 01 Feb 2005 18:20:15 GMT

eddwo wrote:

He mentioned the idea of having lots of different virtual machines sitting on a microkernel, isn't that just what Xen is trying to do?
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/
This is different to VMWare and Virtual PC because it does not require a fully featured OS to host the virtual machines.

vmWare's ESX server is along the same line, it's a bare-metal architecture that inserts a small virtualization layer between the x86 server hardware and the virtual machines.

http://www.vmware.com/products/server/esx_features.html

Stephen

posted by PerfectPhase

Re: Neal Christiansen - Inside File System Filter, part II

androidi — Wed, 02 Feb 2005 11:25:48 GMT

I have some questions to Microsoft:

Why it is so difficult to create even a simple search that is _fast_ (and silent) for finding files and folders by their name. I do not know about you but most of the time I need to find a file or folder by name in the current system - It is very rare that I need to find certain string inside a document. I know I am an exception here, but when doing so much simpler search (fuzzy match string againt filename/foldername string) one would expect the local search to be atleast a somewhat faster than Google query over the network when a computer has only less than million files/folders - certainly something very much achievable by average home computer if its well done.

I know MS provides the Indexing service, but I've tried to figure it out many times, and it has never made any sense to me (I doubt it's necessary to point out where it has failed). And I am not even interested to build my own UI to the indexing service for the reason that it makes a lot of noise during the indexing (Unless you happen to have solid state storage).

I would like to present how simple it is, in theory, to make a silent full indexing:

1. Suppose user introduces a medium not previously indexed (cd/dvd/serial-ata hdd?)

2. As the disc is recognized, it is matched against previous media signatures

3. User loads a file from the disc
3a. The loaded file is entered into the index, which is partially stored in memory to avoid noisy random HDD access and lazily updated with transactions to system HDD
3b. As the user mode process does something with the file(s), in the file(s) content is being analyzed in the background (like your virus scanner does) and proper records created into the memory for the lazy noiseless update.

4. User is done with the file and does no further access in the so far mostly unindexed media.
4b. Incase the medium is CD/DVD or other very noisy one, the appropriate driver is notified to spin down the speed enough for the medium not to make noise during read (in case of CD drive this should be either user or manufacturer definable rotating speed)
4c. The last (before idle/after lowering speed) position of the hdd/dvd is picked up.

5. Now the full indexing can begin from the position of the disc, making no noise that would indicate the media is being indexed.

Certainly there are other questions, like what if user removes the media before its indexed etc - but the key point here is the silent, pleasant way of access - some of which NCQ may help slightly, but in the case of jumping around between file data and MFT - I think not.

One thing that really amazes here, is that Anti-Virus programs pretty much do everything that would be needed to create fast searchable index.. Why they do not take advantage of this, must be because searching the user file contents for certain data is not their core business (Or is it, I am not the expert here).

I can understand that some people may have not yet learned to appreciate more silent environment, but looking at what people say about the new set-top-boxes and other livingroom "PC" equipment with HDD's, you can expect that if they get a Longhorn Embedded on their box and it has noisy indexing, there may not be many positive reviews in that regard. EndRant();

PS. To anyone who cares, certain Seagate SATA HDD's may have annoying feature that writes some data to the HDD that only Seagate can read at their service during idle moments. So incase your HDD will fail, Seagate may have better clue why their certain products . Hitachi has same feature, but where the Seagate noise is like short ranged constant seeking, Hitachi noise is like the drive is going to fail any moment. The takeaway - Do Not Buy if the computer with these HDD's is anywhere near you (<10 meters).

posted by androidi

Re: Neal Christiansen - Inside File System Filter, part II

Mike Dimmick — Sun, 06 Feb 2005 20:20:50 GMT

rhm wrote:

I'd like to see a video of kernel debugging in action using that windbg that was mentioned. You know, just to see how hairy it really is in there.

You want to debug your kernel?

Download WinDBG from http://www.microsoft.com/whdc/devtools/debugging/default.mspx. Recent versions running on XP or 2003 (I don't think this was supported on 2000, but could be wrong) offer a 'Kernel Debug' option on the File menu. Select the Local tab to debug your local machine. You have to run WinDBG using an administrative account - after all, it wouldn't be good for security if ordinary users could debug the kernel!

Local live debugging is a little limited. For full control you need to run the debugger on one machine and have a separate machine to debug. Currently you can use a serial connection (pretty slow) or an IEEE1394 connection (fast). There's also kernel-mode remote debugging but you can't debug boot-time with this option.

(Bad UI, guys! I don't expect a tab to control which option I'm using to connect to my kernel - I expect a set of option buttons).

I'm only a user-mode developer but I keep WinDBG around because it is more powerful than Visual Studio (although newer versions of VS are getting closer) and it's more lightweight. I recently solved a problem in a VB6 app by compiling the app with debugging symbols on, calling DebugBreak explicitly in a test version at the point where I knew the error occurred, and running the app under WinDBG on all machines in a load-balanced cluster. When the error occurred the app broke into the debugger, I dumped the stack, and worked out what had gone wrong.

As always you need a guide to kernel mode. Windows Internals 4th Edition is probably good. I don't have this yet, but I do have the previous edition, titled 'Inside Windows 2000' by the same authors.

posted by Mike Dimmick

Re: Neal Christiansen - Inside File System Filter, part II

Rob_L — Tue, 19 Sep 2006 00:43:46 GMT

The 1959 photo of an early home computer that Neal said he saw was an advertisement for the Honeywell H316 Kitchen Computer. Neiman Marcus tried to sell it to Housewives in early 1960's. It was a primitive computer for storing recipes and shopping lists.

posted by Rob_L

Re: Neal Christiansen - Inside File System Filter, part II

himanshup — Thu, 31 Jul 2008 12:54:51 GMT

hello all

if any one can plz tell me that can i intercept network io in file system filter driver.

plz help me.

posted by himanshup