Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

What Do You Want to Know About Cortana? Ask the Guy Who Led the Cortana API Team, Rob Chambers

Later this week, Microsoft Evangelist Jeremy Foster will sit down with the General Product Manager for the Conversational Understanding Platform, Rob Chambers and ask crowdsourced questions from our community on a topic we're all excited about: Cortana.

If anyone has insight into how developers can get a jumpstart on integrating Cortana into new or existing apps, it's the person that helped define and build the new capabilities available for developers for Windows Phone 8.1. As the General Product Manager on point to harness the power of Bing and bring it to life with Cortana, Rob worked closely with Dev Manager Mike Calcagno to ensure the end product would delight end users and empower developers.

What do you want to know?

Have you been curious about the CU API? Do you want to integrate it into your app? This is your chance to pick the brain of an insider and get your questions answered. Here's how:

Simply tweet your questions to #CortanaQA and we will get as many answered as possible during a taped interview segment on Channel 9 on April 25. If you're already starting to integrate Cortana into a new or existing project tweet your screenshots, Vine clips, and videos of your project using the #IDevThis hashtag where we'll be looking for the next project to showcase on Ch9.

Rob's Guidance for Developers Considering Cortana

To get you ready, we asked Rob to describe his role and give some pointers for developers chomping at the bit to build something great with Cortana.

What team do you work on at MSFT?

 I am the GPM for the Conversational Understanding (CU) Platform and API team inside Bing. The CU team is responsible for all speech and language understanding technologies that other Microsoft platforms and applications use to NUI enable their systems, like Xbox One, and most recently Cortana on WP. The Platform and API sub-team inside CU builds the API surface, and platform components at the top of the software stack that run on the phone itself. These are the APIs that developers inside WP use to build Cortana, and also the APIs that third party developers can use to extend Cortana's

At a high level (since we'll address this in the chat) what do developers need to have ready (or be prepared with, etc.) to integrate Cortana API's in their app?

 There are basically three things developers have to do to integrate with Cortana.

  1. Define what "voice commands" their application would like to support, declaratively, in a schematized XML file. These voice commands include phrase patterns, like "Skype, Call [contact_name]", or be as general as "Hulu, [whatever_the_user_might_say_here]". The voice commands also contain additional information, like how the application should be launched, and what should the user see while the application is loading (e.g. "Searching for Mr. Peabody showtimes...").

  2. Register the XML file containing the voice commands at application first launch. The voice commands can be updated after the fact, as well, but like a Background Task, you must "install" the commands when your app is run for the first time.

  3. Handle the voice activation, once Cortana recognizes that the user spoke your app name, followed by a command you've registered". You'll receive, on application activation/page navigation (WinRT app vs Silverlight app) parameters that indicate which voice command was recognized, the fill_in_the_blank_phrases, as well as the full text that was recognized (or typed) into Cortana's UX.

That's it...

What use cases should developers focus on first?

Applications that help users communicate with people, or the world, Apps that help the user remember or do something later, or applications that help the user find information, or objects, and optionally do something with those things ... All of these application patterns are good application patterns to implement when integrating applications into the Cortana experience. Primarily, that's because these patterns are the most powerful scenarios that help Cortana users the most.

What is the ultimate integration of Cortana you'd like to see?

I'd love for 100+ of the top 500 applications in the store today to integrate with Cortana, especially ones that match the patterns I just described. If I had to pick a single application that I'd personally like to see integrate with Cortana ... I guess I'd say Amazon Mobile, or maybe IMDB. If you would have asked me a couple months ago, I would have said Flixster, but they're already working on that now.

What are you working on now? Does work continue on Cortana or are you jumping to a new, awesome project?

I'm continuing to work inside CU, on the Platform and APIs, working with the same folks I worked with on WP 8.1, trying to take the next big step for NUI for our users. I've been working in or around the NUI space for 15+ of my ~19 years here at Microsoft. We're not done yet, and so ... I'm not done yet either...

 

[UPDATE] Check out the Q&A with Rob Chambers here: http://channel9.msdn.com/Blogs/Dev-Inspiration/What-Developers-Need-To-Know-About-the-Cortana-API

 

Tags:

Follow the Discussion

  • Ian WalkerIan2 In geeks we trust ...

    I would like to see a rough platform roadmap if that is OK?

    EG

    YEAR 1:

    Windows Phone -> US -> EMEA -> Rest of the World

    Windows 8.1 Integration

    XBOX One Integration

    YEAR 2:

    Cortana as a service in Azure?

    Third Party Platforms? (IOS/Android?)

  • @Ian2 - I agree.  I want to see Cortana pushed to Windows 8.x (or 9) and then onto Xbox One.  It would also be cool if the team created a UserVoice page for Cortana so folks could request and vote on features. 

  • I asked here (reddit) but just to be sure:

    One of my apps name is "Quotastic". I register it as CommandPrefix and it works perfectly on 8.0 but when this runs on 8.1 (using Cortana or not) the speech system always try to translate it to "real" words. So I say "quotastic", and the result is something like "old * kicked", "good to see", "go to seek", "old * chick" "odessa texas"(?!), "comcast". I ended up changing the CommandPrefix to "Quote" and it works, but is not ideal. So, whats up? Why 8.1 is not picking up my commandPrefix and giving priority to it when trying to find out what the user said?

  • Vipul S. ChawatheVSChawathe Vipul

    As NUI evolves, the top store apps will hopefully be upset by surprises rather than existing top apps doing better with Cortana. NUI+AI will take UX to unforeseen level. Thnx. :)

  • roschlerroschler Channel 9 Fanatic

    It would be nice if everything was morphed into the Share contract and have that  interface made available/uniform between the Windows Phone 8 and Windows Store 8 platforms.

  • mahamudul haquemahamudul haque

    I downloaded some apps hear all use my tab mobile business program.None job continues got reach.How I make it properly? Use right apps for daily life in my E - commerce.I will be happy in society.Living with lifestyle.

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.