Gordon Bell and Jim Gemmell - A look into Microsoft's Bay Area Research Center, Part I

Interesting segment!

*Thumbs up*

Good to see that Gordon Bell is still active and thriving!

Besides the terrabytes of data... there is the problem of keeping that ever growing set of information moving into future tools and technologies while keeping the formats and links in place at an affordable price for the general public over hundreds of years.

Those full-wall whiteboards are becoming the indespensible design tool de rigeur...

At first I couldn't see it..I mean why would anyone want to persist al that data? My perception was that Bush had the visionary genius in mind when developing his Memex. Like a Buckminister Fuller type, someone who looks at a leaf and sees a new design for a fusion reactor, not an archive of Scoble's droolfest at the new Apple Store (sorry!)

But that real-estate demo with the overlap between multimedia and time definitely clicked.

Okay so text is easy..once its tokenized, indexing and retrieval is a snap.

How do we begin to think about indexing audio and video? Is it even possible?

Anyway, these MSR vids are fantastic! Keep em comin!

Many thanks for this video - good to see something of the MyLifebits work - and to see Gordon Bell again.

I know that I'm getting old when I read that Scoble needs to point out who Gordon Bell is for the sake of the younger generation, and when I hear on the video that Charles did not seem to have heard of Vannevar Bush and the Memex...

Oh, that Gordon Bell.

http://www.itconversations.com/shows/detail371.html

Does the video shake horizontally or is it me.. Well the black bars are gone though, thats good

edit: Guess it was just that the video guy was so excited.. The shaking goes away after a while.

Scoble talks about (around 26-27 min) how people can't do links to the middle of the channel 9 videos. Well I argue that the technology is already here, just needs some refining.


Suppose we have a player plugin that is integrated with the browser. Now when I am the one who decides to watch the whole hour long episode and see interesting segment, I just do what I do currently already, stop the video (I haven't finished it yet) and come talk about it. But with the plugin I can mark the interesting spots in the stream and the browser extension allows me to create a special hyperlink with the timecode. It could also be possible to gather these different links from other people to a central metadata file associated with the video. This way the video author would have people watching the video "do the editing", by giving the interesting timecode segments.

So the tech for most of the problems discussed is here already, just need to code and spread it!

Regarding the clipping of videos, I hate that. The early Channel 9 videos with 5x 1 minute segments just plain sck, as you need a lot of clicking to watch the whole thing and most of the time you want to watch the whole and comment only on some short segments.

It should also be possible to allow people watch to say a live TV broadcast or keynote presentation in the net, in such manner that everyone watching it are approximately in sync and can add their text and audio/video commentary which comes available in realtime to other who watch it. Kind of like commenting a movie with friends in the hometheatre or whatever.. Technically possible to do, just need to integrate it in the player that comes with the OS so that everyone would have it.

Course it would also be nice to record the commentary to some repository so others who watch it later get it too.

So practically MS should integrate parts of the webcast tech to the generic player tech.

Its funny that while the discussion in the video came almost to touch the Google TV, where you can do searches in to the latest TV broadcasts cause of the subtitling, but never mentioned that. Basicly the people who watch the video should be creating the "subtitling" by creating commentary etc.

androidi wrote:

But with the plugin I can mark the interesting spots in the stream and the browser extension allows me to create a special hyperlink with the timecode. It could also be possible to gather these different links from other people to a central metadata file associated with the video. This way the video author would have people watching the video "do the editing", by giving the interesting timecode segments.

I think it's a server-side problem, not client-side. If you say, I want the 22nd minute of this video, the server must know how to get there & start streaming. I know RealMedia does this. I'm pretty sure, Windows Media Server does this. It'd be nice to be able to specified starting point & length as a hyperlink (I don't know if they do or not). So, the solution is there. Maybe MS should proliferate Media Server, too.

I was waiting for this video. Great Job Robert. I was blessed to visit the same facility a two days before Robert http://ipattern.com/simpleblog/PermLink.aspx?entryid=75. I followed Gorden Bell's work for sometime now and finally had a chance to talk and share the ideas of where do we go in industry.

On Tuesday, October 28, 2003  I published entry about where some of WinFX inovation come from.
http://ipattern.com/simpleblog/PermLink.aspx?entryid=10, then I IMed Robert about Gorden Bell because at this time he was evangelist for LongHorn team and I thought it was very important for him to get to know MyLifebits project.

I am pleased that many more people will pay attention to the research work now.

My two cents, Maxim
[www.ipattern.com do you?]

Minh wrote:

I think it's a server-side problem, not client-side. If you say, I want the 22nd minute of this video, the server must know how to get there & start streaming..

Perhaps. However clients can already seek in the stream to a point they want. Just give me the timecodes and have the player do the seeking. This approach will work with every streaming format, not with just MS ones.

For people to use it the clients must be tailored or have a plugin to support it too. I know I would not want to type in some timecodes, I should be able to quickly seek through the stream and create the videolink in one click and add the commentary without leaving the mediaplayer, but the support for this could come in as external to the player perhaps so it would not be tied in to just MS's mediaplayer. Some kind of API for this perhaps that provides the link creating and popups a stylish window for creating the commentary or adding audio comment.

After finishing the video and having added the comments, you could either transform the comments to clipboard for putting in a blog etc or you could create xml with all the metadata, link to video and comments which could be passed on so that others who view the video in full length see the comments too..

androidi wrote:

Perhaps. However clients can already seek in the stream to a point they want. Just give me the timecodes and have the player do the seeking. This approach will work with every streaming format, not with just MS ones.

Without the media server back-end, you would have to download the entire stream up to the point to want to start. For local media, you wouldn't notice it because it's so fast, but with media on the net, you'd have to wait a bit.

Minh wrote:

Without the media server back-end, you would have to download the entire stream up to the point to want to start. For local media, you wouldn't notice it because it's so fast, but with media on the net, you'd have to wait a bit.

I do not quite understand? Its possible to seek in streams through HTTP. It's even possible to do this when playing an Mpeg 4 inside AVI from an FTP site. However this does require client code. From server it only requires that resuming is supported from arbitrary point.

Its even possible to do all this in just very few lines of C#

One thing that wasn't touched on is how people could interact via their archives.  It seems like it would be easy to compare virtual lives and find people with very similar interests.  That's just one example

androidi wrote:

I do not quite understand? Its possible to seek in streams through HTTP. It's even possible to do this when playing an Mpeg 4 inside AVI from an FTP site. However this does require client code. From server it only requires that resuming is supported from arbitrary point.

I'm not really sure how a web server would know where the 22nd minute of a WMV begins. Could you give me an example of a HTTP media that can start at an arbitrary point?

Minh wrote:

I'm not really sure how a web server would know where the 22nd minute of a WMV begins. Could you give me an example of a HTTP media that can start at an arbitrary point?

Well lets suppose I am going to make a link to inside the video, first I'd have to watch it to know where the interesting content is right?

I am not certain it would work, but you could simply just try determine the number of bytes into the stream (could be hard) and then have the videolink contain both the byte pointer and the time index. One limitation would be that the byte pointer would only be valid if the others would use the same stream, so having a 128,256,1024 Kbit streams it would only work for one of them without also watching the others

Other way could be to use estimate where the 22:00 minute is, go there, if its the 30th minute, use some algorithm to jump a couple times in the stream until close to the 22th minute. I am not sure however how much data would be needed to find the right position, however jumping like this to find the correct minute still takes less time than watching the whole stream from the start..

Wow, I want my WinFS!

I read the Memex paper over a year ago, as it was talked about when WinFS was previewed at the PDC. It was exciting to see it all finally becoming a reality. I agree that its all about the links between things, and that full text searching just isn't quite enough.
In some ways I think Apple will get there first on the Mac, as they own not only the platform but the media apps that most people use. They can deliver Tiger and a related Mail, iPhoto, iTunes etc and tie it all together in one go, wheras Microsoft has to build the platform and wait for developers to fill in the gaps.

At the same time things like gmail, and flickr, really show that you can have those sorts of queries and retrieval performance without it being tied to a single PC, I certainly see the benifit of having everything available from several machines running different platforms from wherever I happen to be.

Theres such a lot in that video that I've wanted to do.
I love doing research on the web, where you pick a subject and find loads of related things, and go off on three or four tangents at once. I'd love to be able to come back to that later and see a map of all the sites I visited while looking up a subject.
It might be several pages on wikipedia, an academic paper, a patent application, some news stories, some old usenet posting on google groups etc. It should be possible to automatically build mind maps, that keep track of your different trains of thought.
I don't just mean a record of all the links I followed, because I tend to be more branching than that. I'll usually grab some text from one page, and drop it into google and start another line of enquiry in a new window or browser tab. I'd like to include those in the map as well, and any related resources I downloaded such as pdfs, powerpoints and video clips.

I know things like OnFolio and OneNote, can help you record the contents pages you visited, and of course the browser history keeps the urls, keeping lots of bookmarks quickly gets unmanageable and it doesn't seem natural enough to capture a browsing session as a whole.

It seems like an awesome job, just to be able to come up with new ideas and make prototypes of new products. What would I need to do to go about getting into such a research position?

This is by far the most genuine, flexible and friendly part of MS i have seen, and ive seen all the c9 videos. This is the kind of stuff that uni researchers should be doing; unfortunately many are bankrolled by companies and this can focus them too much on a single train of thought.

I think the success of this group from what I can see is that its small, has good people in it, the people are allowed to do what they want which means dropping everything or putting it on the back burner while moving in another direction.

I remember BT (British Telecom) were trying to win in court the rights to Hyperlinks, they didn't win - so this old guy made them!?!

About the point of searching and indexing specific sections in a video file when using Windows Media Encoder and WM9 there is an option called 'frame-level seeking' which is enabled allows for just such seeking doesn't it?

Samuel

It was mentioned in the video about the lack of ability to link to specific parts of the video. Do links such as:

www.stewartwhaley.com/blog/links/Gorden_Bell.asx

not work for other people?

<Duration value = "00:00:35.080" />
<StartTime value = "00:22:5.812" />

Ah, hey I did not know about this

However my point was entirely that the tech need refining, or can you do these links with some simple way from media player? And it seems you also need a place to host the asx - so essentially not useful without more work.

Thanks so much for this video.  I first heard about this project back around 2002, when it was known as the CyberAll project.
Great to see that there still plugging away at it, as this is something I'd love to see available someday.  As they said even back then, getting everything digitized isn't the hard part.  It's retrieval that is difficult.

Thank you for nice video.
I can see MS researchers are tackling with the next file system especially contents DataBase problems.
We may need a Convenient Binder Tool (CBT) before WinFX.

P.S.
I'll visit Moscone West in SF late March.

Lisa Stifleman did some work at the MIT Media lab

With Real Media it is indeed possible to hyperlink to any part of a long video. I was suprised this didn't come up during the discussion or in the comments.

Here's a 40-minute long video stream in Real Media.

And here's a link to 1.5 min clip from the middle of the video.

You can edit the start and end time to see any part of the video.