Entries tagged with search - Channel 9

How to Install and Use Search Connectors in Windows 7

Sarah Perez — Wed, 26 Aug 2009 14:39:00 GMT

One of the many great new features in Windows 7 is something called Federated Search. As we’ve mentioned before, this new type of search allows you to add “search connectors” that let you perform searches of internet-connected web sites and databases in addition to just searching your local files. It’s a full-on mingling of the web and the machine. Earlier, we pointed you to a link for a zip file of some of Chris Pirillo’s favorite search connectors, and now let’s look at you how you can use them.

First thing’s first – you have to click the above link to download the zip file. If you’re having trouble with the link (it was slow to load for me), you can grab Redmond Pie's pack or the Twitter Search Connector from this site, at the very least.

On the dialog box that appears, just click “Open.” The connector will be installed automatically.

Now – how do you use them? This actually is the easiest part. You can perform searches from within any open Explorer window – just use the search box in the upper-right corner. (You can also start a search from the Start Menu, but here you have to click the “See More Results” link to be taken to an Explorer window where you can begin to use the various connectors.)

Here’s an example of me searching all Twitter posts for the phrase “new Twitter app”:

Notice how my search terms have been highlighted in the results and the results themselves are sorted in chronological order with the newest posts first.

Now if I come across a post I want to see on the web, I simply double-click the entry. That takes me to the actual web page where I can read the original item.

Example - Checking out this tweet about Chirp, a new WPF-based Twitter app:

That’s it! Search connectors aren’t uber-geeky or hard to use at all. All you have to do is click and install them when you find someone who has them available for download on the net.

(Note: Chris Pirillo's pack contains a FriendFeed search connector – a must have for all us social media addicts!)

Canonical and ShortUrl on Channel9

Duncan Mackenzie — Tue, 28 Jul 2009 11:11:00 GMT

One of the things we spend time on for Channel 9 is trying to make sure we are following the right standards, adopting new ones as necessary and that we render correctly for both real people and for search engines. As part of that ongoing work, we recently adopted two new concepts.

Canonical URL

The first is the concept of a canonical URL, as described here by Matt Cutts (his blog is a must read for anyone building public facing web sites); a link element that specifies what our definitive single URL should be for any given post.

Why does this matter? Well, on many web sites (including ours) there is more than one URL that will get you to the same piece of content. Consider the latest 'This Week On Channel 9' episode, it is available at:

Assuming Google/Bing/Yahoo only found the post from links on the Channel 9 home page, it would always see the first one... and everything would be great. That isn't how search engines work though, they care about inbound links from many different sources, and it is possible that many different URLs are out there in the wild that really represent the same single piece of content. Each additional URL beyond the first looks like duplicate content and takes away from the search engine love that should be given to the first result. The standard way to avoid this in the past was to redirect every person coming in on anything but the link we want. That works, but forcing your user to go through an extra browser round trip for some obscure technical reason is less than ideal. Enter the idea of the canonical url. Add this to your page and no matter how the search engine finds the page, it knows what URL to associate it with in the system.

If you check any of those links above and view source, you'll find the same thing on each and every one:
<link rel="canonical" href="http://channel9.msdn.com/shows/This+Week+On+Channel+9/This-Week-C9-Windows-7-RTMs-7-Sins-of-App-Compat--cool-Silverlight-apps/" />

A fair number of other sites implement this as well, check out Ars Technica for example, and I'm sure more will follow. Should you implement this on your site? Well, first think about how many duplicate URLs will work on your content (if you are running a site that supports both www.sitename.com and sitename.com, that's one ... then if you can optionally have a filename like default.php or index.html... then that is already two duplicates for every URL) and then think about whether or not your position in search engine rankings is important... odds are you should look at adding support for this link tag on your pages.

ShortUrl

A lot has been written about how services like TinyURL, is.gd and other URL shortening services are bad for the internet, and we completely agree. They remove meaning from the link you are about to click (including the source of the content, which is an important issue for trust and security), and they are dependent on the reliability of some unknown third party that might just go away at some point in the future. One solution, that sites like C9, Amazon and others have decided to go with is to implement their own URL shortening. Yes, much of the meaning is lost, but at least they have control over that URL namespace and can make sure it is always available and always points at the intended content. Now that we have such a service though, what's to stop people from just taking our original URLs and using any one of the free URL shortening services? Well, nothing at the moment, but a movement is underway to allow content owners to specify a pre-existing short URL if they have one. The hope is that once this concept catches on, then URL shortening service or client applications (like twitter clients) will try to look up a site's short url before calling out to a 3rd party service to create one. We don't know if this will catch on, but we like the idea so we've gone ahead and added the appropriate link tag and populated it with our special r.ch9.ms short url. Once again, if you view source on that TWOC9 episode from above, you'll find this:

<link rel="shorturl" href="http://ch9.ms/AAPV" />

More to come...
If you went and viewed the source of those pages, you would have seen a ton of other <link> tags, and many different <meta> tags as well. Each of those does serve a purpose, and I'll dig into the rest of them in upcoming posts.

Note: For those of you who have the immediate response of 'aghhh... so much wasted bandwidth for meta tags!', I know what you mean... and I also know that some sites choose to only render those tags out when a search crawler hits them. That practice is a bit sneaky though, in general you should avoid alterting the content you serve up to search engines... although it is probably only an issue if you change the actual content of the page in an attempt to deceive the search engine.

Expert to Expert: Harry Shum - General Purpose Search, Decision Engines and Bing

Charles — Wed, 01 Jul 2009 03:44:00 GMT

Harry Shum is the VP of Engineering for Bing, Microsoft's latest search engine offering (well, it's more than a search engine - it's a so-called Decision Engine, but what does that mean, precisely?). Harry has a long history in the world of complex algorithm design and implementation. Before joining the Bing team (at Bill Gates' request), Harry was a reseacher in MSR specializing in computer vision, which is an algorithm instensive discipline rife with machine learning principles, statistics and in some sense artificial "intelligence" in terms of autonomous pattern recognition capability. At any rate, Harry is a developer and scientist through and through. We're very fortunate to have him running our search engineering efforts. General purpose search is an incredibly fascinating area with a great deal of potential, challenges and opportunities.

Erik Meijer, programming language designer, knight of the lamda calculus and Expert to Expert host, sits down with Harry to learn, at a high level (though deep in context), how Bing works, what, exactly, a decision engine is, what really happens when you Bing something and various topics related to the computation behind both general purpose search and accurately interpreting user intention. Of course, being an E2E, we take the conversation in many directions and Harry was a real sport. Thank you, Harry!

If you want to know the past, present and a little bit of the future of Bing technology, well, tune in and meet Harry Shum; a computer scientist, software developer and vice president.

Enjoy!

Ping 14: Bing, VPlay, Zune HD, Celebrifeed, Mediaroom

Laura Foy — Mon, 01 Jun 2009 13:24:00 GMT

While Max is away our dear friend Paul stepped in and he definitely wasn't shy to voice his opinion. He's got his (very long) finger on the pulse of Microsoft...these were the beats:

And Interview with the Bing team
Bing goes live!
VPlay for Surface DJ's
Are you ready for Zune HD?
Paul schools Laura on Mediaroom
Celebrifeed for Hollywood Twits

...and a challenge for some badass giveaways! Watch and play...

Bing: Test Drive Microsofts new search engine

Laura Foy — Fri, 29 May 2009 20:05:00 GMT

At the All Things Digital Conference, Steve Ballmer just announced our new search engine called Bing. Bing is actually MORE than a search engine- we are referring to it as a "decision engine" as its goal is to help users make more informed decisions by collating and formatting information for you that is relevant to your search. It's exceptionally useful in areas like travel, shopping, weather and movie listings. Beyond there's there are a ton of new fun features and time saving capabilities. Watch as Stefan Weitz takes us on a test drive.

Read what the critics are saying...

Stefan Is… researching the art (or is it Science?) of Search

Max — Tue, 31 Mar 2009 18:02:00 GMT

Believe it or not, Search is a relatively new technology. You probably perform a few searches a day… how satisfied are you with the results?

Stefan and I sat down with Martin, a Senior Product Manager on the Search team, to find out the process for improving search and where things may be trending. An interesting discussion with the slickest sticky-note you’ve seen today ;).

Xin Wang: Science Fiction, Artificial Intelligence and Search

Charles — Tue, 17 Mar 2009 20:39:00 GMT

Meet Xin Wang, SciFi fan and an expert in artificial intelligence (she has a Ph.D in it!). Today she works on Markov decision problems and how to use approximate optimization problems that can be traced all the way back to Sir Issac Newton to solve combinatorial optimization problems. Xin has worked as an SDE at Microsoft for more than 4 years; when she first started she worked on a very cool project for ink parsing on the Tablet PC. Watch the video to find out more about this very interesting software developer!

IE8: Search

Charles — Mon, 23 Feb 2009 22:28:00 GMT

Sharon Cohen, Program Manager on the IE Team, sits down with us to discuss some very interesting (oh, and useful too) new Search features in IE8. Tune in.

Jim Mosher: On Interaction Management and the FAST Search Engine

Charles — Mon, 05 Jan 2009 19:10:00 GMT

Jim Mosher, Senior Program Manager in the Microsoft Enterprise Search Group, sits down with us to discuss Interaction Management.

Interaction Management enables users to interact with and consume information more quickly and more effectively through search and search-related technologies. The Microsoft Enterprise Search Group (MESG) is driving this initiative, with plans to provide enabling technology initially to the FAST search engine (Microsoft purchased FAST last year) and eventually to a broader application base. This conversation provides an overview of our vision for Interaction Management and points out where Interaction Management concepts are being used today.

CRM Accelerators: Enterprise Search

Girish Raja — Wed, 19 Nov 2008 20:33:00 GMT

In this video, we take a look at the Enterprise Search Accelerator for CRM and see how SharePoint and CRM can complement each other in creating business solutions.

The enterprise search accelerator allows Microsoft Office SharePoint Server (MOSS) customers to view and search for Microsoft Dynamics CRM data directly from their SharePoint portals.

Additionally, the technology, Business Data Catalog (BDC), and techniques used with this accelerator can be employed to surface data from other line of business applications to further enrich the SharePoint portal.

BDC makes it easy for people to connect to, find, and act on information stored in structured line-of-business systems (such as Microsoft Dynamics CRM) by using a declarative framework to securely integrate them into search results. With BDC you can configure actionable audience-specific portals, dashboards and mash-up interfaces for this data without writing any code.

Adding Search to Outlook Context Menus

John Wiese — Mon, 13 Oct 2008 21:55:00 GMT

Outlook 2007 added instant search, but why should you have to type in the same query over and over? You shouldn't, so how about an add-in that adds your common searches to the context menu in Outlook. In this screencast I'll show you how to add some simple searches to the context menu so you can access those searches when ever you like, without having to type them into the instant search box.

Scott Prevost explains Powerset's hybrid approach to semantic search

JonUdell — Fri, 26 Sep 2008 09:12:00 GMT

Scott Prevost is General Manager and Director of Product for Powerset, the company whose semantic search engine was recently acquired by Microsoft. In this interview he describes the history of Powerset's natural language engine, and explains how it works as part of a hybrid approach to indexing, retrieval, and ranking.

Scott will expand on these topics in his keynote address at Web 3.0 in October.

Scott Prevost

JU: The notion of search enhanced by natural language understanding has a long history. I was just reading Danny Sullivan's rant about how he's been hearing about this for years, but it's never amounted to anything.

Of course, people are all over the map on this topic, but nonetheless you guys are doing certain demonstrable things, and working on other things. So I'd like to find out more about how the technology -- which was acquired from Xerox, where it had been worked on for a long time -- actually works. What you mean by natural language understanding, how you're applying the technology, and where this is going.

SP: Well, there are a lot of questions tucked in there, but maybe we can start with what we licensed from PARC, what was formerly Xerox PARC. They had been working for 30 years in a linguistic framework called LFG -- lexical functional grammar -- and they built a very robust parser. It's probably parsed more sentences than any other parser in the world.

What it allows us to do is take apart every document that we index, sentence by sentence, uncover its linguistic structure, and then translate that into a semantic representation we can encode in our index.

JU: Can you confirm or deny something that Danny Sullivan reported, which is that it takes on the order of two months to index Wikipedia one time using this method?

SP: [laughs] That's a very, very old number. It all depends on the number of machines, but we do it now on the order of a couple of days.

JU: And it scales linearly?

SP: Yes. And in fact we're working really hard to bring those numbers down. We have a very small data center right now. We're looking at what it takes to stand up a 2 billion document index, and it's absolutely attainable.

I think Danny Sullivan realized, when he wrote another article on the day we launched, that we're doing something different. He called us an understanding engine. It's not the case that we're just applying linguistic technology at runtime, by parsing the query and then trying to use the same old kind of keyword index for retrieval. We're actually doing the heavy lifting at index time.

We're actually reading each sentence in the corpus, pulling out semantic representations, indexing those semantic representations, and then at query time we try to match the meaning of the query to the meaning of what's in the document. That allows us to both increase precision and improve recall.

JU: When you say semantic representation, what it means -- or anyway what's evident in the current version -- is subject/verb/object triples, basically. That seems to be how things are organized.

SP: That's one small part of what the engine does. It's the part we've exposed in the user interface in a very direct way. But actually those are only three of several dozen semantic roles that we uncover at index time, and all of those roles go into selecting documents, and snippets of documents, when we present the organic results.

JU: Really? So even though the patterns aren't exposed in the advanced query interface, they're still used?

SP: That's right, they're still used.

JU: What would be an example of one of those other patterns, and how it's applied?

SP: So, you ask a question like: "When did Hurricane Katrina strike?" The 'when' is a certain kind of semantic role that we've indexed, separately from the subject, verb, and object. There are a number of other roles like that: location, time, other types of relationships.

JU: I saw a private demo, about a year ago, in which one of the most striking examples was something like: "Companies acquired by IBM between 1996 and 2003". At that point, I think the light bulb goes on in people's heads about what this could really be.

That class of query isn't exposed yet, but it's an example of what's possible, right?

SP: Absolutely. That's exactly the direction we're moving in. Initially most of the work we've done has been on the index side. Now we're starting to catch up on the query side, which allows us to complete the loop and do queries like that.

JU: The other piece that's visible on the website, in addition to the Wikipedia stuff, is the Freebase material that you've recently integated. That's an interesting case because there you can pull semantics directly from Freebase. So this becomes more of a query-time interface to something which is already structured and understandable.

SP: Yeah, that's right. Freebase is kind of like Wikipedia, except it's all structured data. Unlike with our core technology, which turns unstructured data into structured data, with Freebase we just go directly to the structured data. But it uses the same linguistic technology on the front end to parse the query, which then gets mapped into a Freebase database call.

But by using linguistic technology to parse the query, we're able to match very flexible ways of saying things. We don't have to imagine every possible way someone might ask for a particular piece of information. The linguistic engine takes care of a lot of that for us.

JU: That's why I can type in something like "Barack Obama's book" and get back the answer Dreams From My Father directly from Freebase.

So, what was the intent of including Freebase along with Wikipedia. What are you trying to show there?

SP: That the linguistic technology can be used with both structured and unstructured data. Freebase just has a lot of really great information.

One of the things about a natural language front end is that it encourages people to ask questions and expect answers. With the Freebase database, it's pretty easy to provide direct answers right at the top of the search results page, which users find to be a nice experience.

Of course you have to be very high-precision, so we've tuned the Freebase stuff for precision rather than recall.

JU: Tell me about the natural language landscape: the variety of approaches that exist, the style that you're using, how that compares to others, how all this fits into the history of the technology.

SP: The technology goes back a long way, three decades or so.

JU: Longer, actually.

SP: Yeah, really since the beginnings of AI people have been trying to use computers to understand and generate language. There have been a number of different approaches: purely symbolic approaches, statistical approaches. We really use a hybrid.

The Xerox technology uses a particular grammatical formalism, and we do use symbolic approaches to our semantic rules. But we also then put these semantic features into our index, and use machine learning and statistical approaches to retrieve and rank results.

It really is a combination. We try not to be religious about these things, but just use best of breed, and choose the right tools for the jobs we're doing.

JU: One of the things that Peter Norvig at Google is always saying is that the real secret to their success is vast quantities of data, and that in the end you don't really need AI, you just need lots and lots of data and the ability to crunch through it.

I assume you would argue that the natural language techniques are also helpful, and that as the quantity of data in your possession grows, the power that it brings to the table will also grow.

SP: Yeah. One thing we try to do with the natural language technology is give a leg up to the statistical and machine learning approaches. If you look at a search engine that just uses keywords, the information you have about the page is pretty slim.

We're trying to capture more information about each page that we index, which enhances our ability to retrieve and rank. For example, it allows us to retrieve documents where there are no keyword matches, but there's a good meaning match.

JU: For example?

SP: So, consider a query like: "What politicians were killed by disease?" Powerset will retrieve documents that don't include the words 'disease' or 'politician' or 'kill', but that are about particular politicians who died from particular diseases.

JU: Is the process of mapping generic terms to specific instances a hybrid of human editorial effort and statistical techniques?

SP: Yeah. We use things like WordNet, which is a giant dictionary or thesaurus of the English language that shows how various word senses relate to each other. We use that with some editing on our own. We also use machine learning techniques to figure out some word relationships, and which are most helpful in retrieval and ranking. So it really is a combination.

JU: When did you start this work?

SP: The company was founded three years ago, and I joined two years ago. But of course the work at PARC goes back 30 years.

JU: You obviously have an academic background in this field.

SP: Yeah, I have a Ph.D. in computation linguistics, as do probably about twenty other people at Powerset.

JU: What's your take on how this engine will start to surface through the various Microsoft online properties?

SP: The two areas where we can make a big impact are, first of all, improving core relevance, which is an absolute must for every search engine. And then also the user experience. Some of the technology -- and you start to see it in the Wikipedia search engine that we put out -- some of it really allows us to do different things in the presentation of these results. Thing that can save the user time, by getting the answer right on the search results page.

Our goal is to continue to work on improving relevance, and we've shown that by using these semantic features we can drive large relevance improvements, but there's still a lot of work to be done there.

JU: In that case, the improvements would be under the covers, the person using Live Search wouldn't know that you were contributing to the relevance of the result.

SP: That's correct. Another way it can happen is by creating a different quality of snippet or caption, things that highlight the parts that match the query instead of just bolding the keywords. Actually highlight the answer right there on the search results page, so you don't have to click through to determine if it's the right page.

JU: There's a related area called entity extraction, and there's been a lot of action there. For example there's a company called ClearForest, recently acquired by Reuters, which has put a lot of work into entity extraction. What's the story on that front?

SP: A lot of companies are working on this, we have our own in-house effort for name recognition and entity recognition, and this is of course really helpful as a kind of light semantic layer. But for us, it becomes deeper because we can start to relate all kinds of entities to one another, based on where we've seen them, and also with the help of things like Freebase.

To follow up on how you'll see the impact in things like Live Search, beyond the improvement in relevance and in the quality of snippet, I think you'll see features like related searches, other ways of presenting information similar to the Factz that are shown in our Wikipedia product, I think you'll see a lot more work on the instant answers, with a database that extends beyond Freebase.

Without committing to particular deliverables, these are the kinds of things I think you can expect to see. And you'll also continue to see growth on powerset.com, where we can be a bit more daring in terms of ways of presenting search results.

JU: Well thanks for your time. This has been interesting, and I'll be fascinated to see how things unfold over the next few years. I've got a feeling you'll have access to a pile of resources to work with...

SP: Yeah, we're really excited about it. As a startup, it's hard to build a full-scale web search engine. Having the resources available, and the really smart people at Live Search, is just a tremendous boost to us.

TechFest - Merrie Morris - Collaborative Search, Co-Search, and Search Bar

Dan Fernandez — Tue, 04 Mar 2008 17:23:31 GMT

In this video, you'll see demos of some unique ways to alter the search experience including

- Collaborative Search - searching and annotating with multiple people
- Co-Search - multiple people using the same computer with multiple mice and mobile phones for input devices
- Search Bar - Enables a single person to keep track of their search results

Rico Malvar: Microsoft Research, TechFest, and the future of technology

Dan Fernandez — Mon, 03 Mar 2008 16:53:13 GMT

With Techfest just a couple of days away, I caught up with Rico Malvar, the managing director of Microsoft research and ask him about the history of MSR, some interesting projects coming out at Techfest later this week including projects like contextual search and Worldwide Telescope, and how he sees technology evolving in the future.

Marc Mercuri on platform incubation and Tafiti

JonUdell — Tue, 21 Aug 2007 18:20:59 GMT

Today on Channel 10, Larry Larsen interviews Marc Mercuri who announces and demonstrates an experimental Silverlight-based interface to Live Search called Tafiti.

Marc works on the platform incubation team. In this podcast we discuss what platform incubuation means, how the Tafiti project exemplifies it, and what the future may hold not only for Tafiti but for a platform that's evolving to encompass both software and services.

How-to: Query Vista Search From Your App

jmazner — Tue, 20 Jun 2006 20:39:10 GMT

Ian Griffiths, dev trainer extraordinaire, recorded a series of "how-to" videos for us, each one demonstrating how to write code that takes advantage of a feature of the Windows Vista platform. Here's the first in the series -- how to issues queries against the desktop search engine.

[Update: the second installment, Windows Live 'Hack Day', gather developers together into one place, feed them and set them loose trying to build as many cool apps as they can.

The Voice of Support - Show 002: Search @ support.microsoft.com

gsamedia — Tue, 11 Apr 2006 15:20:14 GMT

Welcome to the second installment of The Voice of Support!

In this second show, Ken Crawbuck and Beth Phillips from Global Service Automation (the team behind http://support.microsoft.com) cover Search on the Support site - info on usage, tips and tricks, some background on algorithms, as well as what to expect in future versions of search, which are making their way to the Support site very soon.

While we certainly have a lot of support-related content ideas to bring you, we really hope to hear from you, our listeners, about the specific topics you'd like to hear about in future sessions! Please use our Ask for It form to let us know what you think!

LINKS

Download the Transcript

Microsoft Support WebCasts

ASP.net Support Voice Columns

Andy Edmonds and Erik Selberg - Frank talk about MSN Search

scobleizer — Fri, 21 Oct 2005 21:14:02 GMT

Andy Edmonds, lead program manager, and Erik Selberg are two of the geeks who work on making MSN Search better. They talk about how the engine works, and what they're doing to beat the competition. Watch me give them a bit of heck about being behind the other major engines. You'll enjoy this frank talk from the team.