Sam Druker - Answers Channel9 questions about WinFS

i don't understand the obsession with email. i'm a home user.  my email doesn't bother me.  Could we use some better examples than email to promote WinFS

For a lot of people, email is the focal point of their daily lives.  It basically controls the workflow and distribution of data, documents, and scheduling.  Even at home, email is the primary point of contact for me for most of my information. 

It's no coincidence that way back in the early 90's Lotus Notes became so popular.  The market was calling for a networked, hyperlinked, enterprise messaging system.  Mail was very much at the heart of Notes.  

Granted there was a lot of over head with it, but that really started the trend of email becoming such a powerful force in business.  Prior to that, most people checked their email once or twice a day, becuase the mail clients were resource hogs. Keep in mind that the most coveted PC at the time was a 486/66mhz and we all remember how great Win 3.1 was for resource management/multi-taksing.

It would appear that a major driving force behind  WinFS is to take this information and integrate it into the file system.  In the future, the primary point of contact for personal/business/etc information will be the file system itself.  The file system is where all the data ends up being stored anyway.  I mean how many times have you detached a document, saved it to the hard drive, deleted the email and then forgot where the file was?

This is a huge jump forward, as email has its weakpoints and was originally designed for a mid 90's kind of world. (ie pre spam, email worms, and assorted mail-based nasties)

At first, I didn't buy into the idea of the file system sitting on top of a database engine, but I'm warming up to it quickly the more I think about it.

ok sure but i can't relate to all this email nonsense.  Sure maybe as a system analysis i'd say "look you guys, Longhorn does this, and i believe that to cut down on time then you should get this".  but i'm not. 

I'd just like to see more scenarios than the age old email one.  it doesn't relate to me at all. 

I think this is promising too much and I doubt it can deliver. It has a greater chance of wasting CPU/HDD/RAM than to actually deliver useful information.. It is like a super-hyped version of the indexing service really, except now your computer stores information without you knowing. I want to see where they draw the line between redundant information and useful information. It also has vast security concerns because it can store duplicate copies of part or all of documentation that you may have deleted.

I find it curious that all the content on WinFS I have seen talks a lot about how it works and what it stores but I am yet to see actual uses for the information. I mean integrating an E-Mail search with the shell is no big deal but then how do you go from that to integrating it with the file system? The same is true about pictures and word documents. You could easily justify the need for a more advanced search built into the shell that supports all of this but a new file system is wasteful and useless.

Longhorn has some things I would like to try but this is not one of them.

Manip wrote:

Longhorn has some things I would like to try but this is not one of them.

I fight this all the time. I think we've had a hard time really explaining the power of an integrated storage platform and we'll need to show folks like you the really interesting apps on top of it to prove our point. I will certainly accept this criticism, and I hope that when you see what ISV's build on top it and what users can do with their data you'll think otherwise.

I suppose I may have overemphasized mail (including news, RSS posts, fax and IM message) in this post. One reason is that I've seen previous discussions around WinFS that are very photo centric or mp3 centric. I talk about the messaging centric viewpoint because I'm using that as a proxy for talking about information relating to People in general.

It's the People scenarios that I think really make WinFS a compelling world for users (not just ISV's).

Frankie's right about this being a long standing wave of computer adoption driven by collaboration. Lotus Notes was a great waypoint in that arc, along with Usenet, BBS's, Bix/Compuserv, ITS NOTES, message boards, RSS and all the various incarnations of mail from uucp to modern SMTP/MIME. People love to connect with each other.

samdruk wrote:

People love to connect with each other.

Sounds like you've read the Clue Train Manifesto.

This kind of skepticism is natural for a new feature. I have to admit, I was one of the skeptics, but the more I think about it, the more the idea is growing on me.  I'll wait to see an implementation before I pass judgment.

Frankie? Wait, you're the guy that posted the British pub rules thing. You owe me a couple hours back from my life; and exactly where was that article when I needed it?

I worked in a London (well, Woolwich) based startup back in the late 80's. I lived in Boston, but spent many a night in some dump of a hotel across the pond contemplating just heading back straight into the office after hitting the pub because it was more appealing doing all-nighters rather than figure out the arcane rituals required to obtain the only substance that made life reasonable for a geeky unsocial 20 yr old so far from home: real beer.

Thank you for assuming I'm hip, but I haven't read The Cluetrain Manifesto, at least yet.

I think there are some great possibilities with WinFS.

I do have some questions though.  Firstly, will WinFS be mostly a technology that requires coding/APIs or prompting to use?  I say this because of the email example.  I can understand that if you are dealing with Outlook or some other Email application, it would be easy to extract the metadata about an email.  However, how would it work if, say, you are using a web based email service (and let's say something other than Hotmail so as to rule out a Passport link).  Does the WinFS try to extract relation information and metadata from the website?  Or will it just not pick up any information?

My other question is concerning the transfer of files which you dealt with, a bit!, in the video.  Its unclear about how the metadata of the WinFS is stored, and gives the impression that the metadata is stored per file.  Surely, WinFS could only work if the metadata is stored with the user and is the user's personal metadata about that file and its relationship with other parts of the system.  I say this because some files will at any time be used by tens of thousands of other users.  For example, a networked Word document.

I'm also curious about how it deals with breaks in WinFS data through sending files or emails and the sort.  As an example, if I send a mail to support@nwtraders.com and then at their end it goes into a call management system as pretty much every major organisation has.  They then reply to me about the issue, not quoting my initial mail and the reply subject is "Reply to Call 185757".  I send them another mail on the issue, etc etc etc.  The way that WinFS has been explained to date is that if I used search to find all information exchanged on this issue, for my boss for example, I'd expect it to return any files and emails regarding this issue.  It seems relatively easy for email or file exchanges to get "broken" if they leave systems to systems not using WinFS and not relate to each other and thus return an incomplete search.

Is there any technical resources describing WinFS. Diagrams showing layout and security separation etc..

I personally can see thousands of reasons for WinFS even at a file system level. about 20k reasons.

I do Genealogy. So dealing with information is a daily deal at home. 

So just looking at winfs from a point of view of handleing different contact info and being able to search on that is a godsend.

An Email exchange. a Newsgroup entry. Even a web page entry.  An VoIP interview.  a recorded interview. Thousands upon thousands upon thousands of photos. with multiple people in them. 

For Me WinFS and its implementation of a Relational (topic map, or even development and supporting ontology like capabilities at the system level)

I can further see the capability for domain specific systems.  Where the OS understands Legal documents in the legal field. or Birth Death Genealogy reports in a home or even professional genealogiest system. Health and legal health in a mediacal office. and on and on. 

Each and every one of these have a basic need for understanding contacts and events.  some have need for location.  Some need to be extended to ensure privacy Hipa(sp) for medical for instance. 

I would say (since I have been waiting for the object based filesystem since it was announced in 95 time frame) that it is finally nice to see an implementation close at hand. although it has evolved over the years. 

unfortantly we are still tied to a modern date and calandar system though from every thing I am seeing. no BC date support out of the box.   

Well i guess that my question will be pretty easy to answer... so get ready for the tough one..... are you ready?... here we go....


How advanced will the WinFS search capability be? Because i heard that you will be able to browse through the partition/hard drive, faster then you are in Windows XP? As in, if i had an jpeg file on another hard drive that was plugged in USB, how long do you suspect that it will take for windows to find it?
 
  Edit:
   (and yes, the hard drive is filled, about 1GB of space left..hehe)

Nice vid. Thanks Sam for answering the Q's off the blog/forums. Certainly a great way of doing these interviews for such technologies as WinFS

Manip wrote:

Is there any technical resources describing WinFS. Diagrams showing layout and security separation etc..

The center of official advance information is here:

http://msdn.microsoft.com/Longhorn/understanding/pillars/WinFS/default.aspx

Cider:

WinFS is a developer platform that will encourage application developers to write code to enable their data to be shared in the WinFS store. Document centered pre-LH applications can store their documents directly into the database (via good old win32 directory/file names) and we have a service that will promote metadata out of those files into our Items. The basic unit of data in WinFS is an Item.

We have a rich platform for end users to build filters (query) over their data and attach actions to those filters--this is the WinFS Rules (nee InfoAgent) platform. Think Outlook rules, but over all kinds of data not just email.

We think of lots of kinds of data that resides elsewhere that gets projected, or synchronized into the WinFS store. Your Outlook data is really say Exchange or IMAP data that sync's into your WinFS store. In that way, your Hotmail data could be sync'd into the WinFS store (but you'd have to use a client application to do the sync, not just visit the web pages). This is similar to how OE and Outlook build "connectors" today. Once that data is projected into the WinFS store, you can build relations and search and query over it just like any other data.

From the point of view of WinFS proper, there is no such thing as "metadata" per se. Our Items are really just data. Some of those Items correspond to win32 streams that undergo promotion--the data records that are promoted are what you might think of as metadata in this respect (like EXIF tags for JPEG streams). However, for a native WinFS application, like OE, the Items are all data. Contact has name, address, email, phone, etc., Message has a recipient list, a subject line, a body, etc.

It is hard to maintain that set of relationships between disparate sets of data. In that respect, what WinFS offers is a great tool set for applications to work with. It's easier for a mail app to keep track of a conversation thread (and the user's desired view of it, and the user's desired Rules that act on it) in the database. WinFS provides the standardized schemas (with Items, Extensions and Relationshis) to model the data and provides some first-class serialization and synchronization capabilities as well. Of of which is more than your poor friendly neighborhood ISV gets to use today when they're trying to solve this in their custom flat file, ISAM or at best MSDE solution today.

OK, sorry for the long post there.

samdruk wrote:

Frankie? Wait, you're the guy that posted the British pub rules thing. You owe me a couple hours back from my life; and exactly where was that article when I needed it?

I was an expat in Frankfurt, Germany for 18 months and there's an odd kind of American-British rivalry/comradery amongst British & American expatriates.

Some pubs in Frankfurt were distinctly British and there were completely different rules there.  Many funny tales of confusion could be told, but I'll save that for my blog.

Since I was only a two hour flight from the UK, I ended up headed to London a lot.  I, too, was amazed that was no table service, especially places that had tables and served food.

Before I looked like too much of an idiot, my friend called me and I told her I was in this pub and I was waiting for 15 minutes for someone to even hand me a menu.  She laughed and clued me in.

I've skimmed through parts of the The Clue Train Manifesto and it's an interesting read.  It's available online as well. 

So...to bring this up a few levels...to give it an elevator pitch...would you say that WinFS is the file sys. equiv. of the registry being introduced in Win95?  If that's not an accurate analogy, do you have one?  As I work for an ISV that makes its living from file systems support.......winFS has some neat potential...however, WinFS currently confuses me more than it intrigues me... 

amg wrote:

WinFS is the file sys. equiv. of the registry being introduced in Win95?

I've been searching for an analogy to describe WinFS to colleagues. Now, I have one.

At first, I wasn't thrilled at the idea of a centralized Registry, because it could so easily be corrupted and blow out everything.  This happened to me more than once in Win95.

But as I worked with it, I took to the idea.  INI files were a scattered mess and I began to like the centralized repository model. As the technology got better, it became more reliable.  I suspect we'll see a similiar disdain to admiration curve with WinFS.

I have a question I've been meaning to ask: NTFS will still be supported, right? We won't be forced into WinFS right away? And will I be able to convert FAT32/NTFS partitions over to WinFS? 

Another question, now that the file system sits on top of a database, will support clustering and storage area networks be built in?

Frankie Fresh wrote:

I have a question I've been meaning to ask: NTFS will still be supported, right? We won't be forced into WinFS right away? And will I be able to convert FAT32/NTFS partitions over to WinFS? 

I think you have the concept inverted.  WinFS is a relational store that is backed (mostly) by a file system (NTFS).  For example, a word doc doesn't get put into a database table along with its metadata.  The doc resides on the file system (otherwise old APIs would break when looking for the file), the doc's metadata is what's persisted in the relational store.

WinFS sits on top of NTFS, not the other way around.

- rc

Well, I feel better...the last two posts cleared up every bit of confusion in my simple mind. =)

The registry is one of the most powerful innovations that Windows has brought to the PC.  Without the registry, there is no sensical way to link software.

The practical implications of a registry for the file system are limitless...and will only get better with time (e.g. the existing registry).

The future looks mighty fun... =)

NTFS is not disappearing, absolutely. We provide some nice interop with file based applications (as mentioned above) and provide a very solid platform for new application development.

Let me make one small correction: word docs reside on disk for versions of word that still call fopen/fwrite/fclose. New applications have a much richer set of functionality in Items to draw on. I can well imagine applications that break up their monolithic file streams into first class entities. Like a slide show app that models the slide show as a list with relationships to the individual pieces of media and annotations thereof. Imagine that some pieces of your slideshow (opening and closing credits, for example) are in a library of slides and can be shared amongst slideshows. Applications where these pieces of organization are available because their underlying structure is schematized and that schema is standardized, discoverable and usable by other applications.

Another way to think about what the future looks like is to think about the plug-in models applications use. In the WinFS world, Items come with a rich programming model: starting with all that we get from the CLR and also replication services, rule processing for users, schematization and reflection, as well as mechanisms for sharing the data. It will be trivial, for example, to write an application that prints mail messages for you, or more importantly to write 15 apps that print mail messages for you to choose from. Those apps just provide views over System.Storage.Message and they don't have to care if those message items were populated by OE, Outlook, Eudora, Notes, or Thunderbird.

A trivial example? Probaly, but it demonstrates the power of sharing the fundamental data that apps manage for users.

amg wrote:

The registry is one of the most powerful innovations that Windows has brought to the PC.  Without the registry, there is no sensical way to link software.

That's an interesting quote since with the birth of the dotnet fx and (config), we've more or less went back to ini files, just with angle brackets. : )

msft discourages using the registry as it dampens the deployment story


- rc

Everything is cyclical.

Jeremy's earlier weblog posts and some of the things he links to are essential reading for WinFS today.

I was talking a few months ago with some folks about Genealogy apps as an interesting vein to mine for showing off WinFS power.

For those of you with a morbid interest in RSS, you should check out the RSS feeder that the Windows Communication History team built as a sample for the WinHEC release. Jeff and the guys are building a pretty cool infrastructure for keeping the personal in computers on top of WinFS. Check it out:

http://blogs.msdn.com/jeffrey_wall/archive/2004/05/05/126836.aspx