What Do You Want to Know About Cortana? Ask the Guy Who Led the Cortana API Team, Rob Chambers
- Posted: Apr 21, 2014 at 11:59AM
- 5,051 views
- 6 comments
Loading user information from Channel 9
Something went wrong getting user information from Channel 9
Loading user information from MSDN
Something went wrong getting user information from MSDN
Loading Visual Studio Achievements
Something went wrong getting the Visual Studio Achievements
Later this week, Microsoft Evangelist Jeremy Foster will sit down with the General Product Manager for the Conversational Understanding Platform, Rob Chambers and ask crowdsourced questions from our community on a topic we're all excited about: Cortana.
If anyone has insight into how developers can get a jumpstart on integrating Cortana into new or existing apps, it's the person that helped define and build the new capabilities available for developers for Windows Phone 8.1. As the General Product Manager on point to harness the power of Bing and bring it to life with Cortana, Rob worked closely with Dev Manager Mike Calcagno to ensure the end product would delight end users and empower developers.
What do you want to know?
Have you been curious about the CU API? Do you want to integrate it into your app? This is your chance to pick the brain of an insider and get your questions answered. Here's how:
Simply tweet your questions to #CortanaQA and we will get as many answered as possible during a taped interview segment on Channel 9 on April 25. If you're already starting to integrate Cortana into a new or existing project tweet your screenshots, Vine clips, and videos of your project using the #IDevThis hashtag where we'll be looking for the next project to showcase on Ch9.
Rob's Guidance for Developers Considering Cortana
To get you ready, we asked Rob to describe his role and give some pointers for developers chomping at the bit to build something great with Cortana.
What team do you work on at MSFT?
I am the GPM for the Conversational Understanding (CU) Platform and API team inside Bing. The CU team is responsible for all speech and language understanding technologies that other Microsoft platforms and applications use to NUI enable their systems, like Xbox One, and most recently Cortana on WP. The Platform and API sub-team inside CU builds the API surface, and platform components at the top of the software stack that run on the phone itself. These are the APIs that developers inside WP use to build Cortana, and also the APIs that third party developers can use to extend Cortana's
At a high level (since we'll address this in the chat) what do developers need to have ready (or be prepared with, etc.) to integrate Cortana API's in their app?
There are basically three things developers have to do to integrate with Cortana.
Define what "voice commands" their application would like to support, declaratively, in a schematized XML file. These voice commands include phrase patterns, like "Skype, Call [contact_name]", or be as general as "Hulu, [whatever_the_user_might_say_here]". The voice commands also contain additional information, like how the application should be launched, and what should the user see while the application is loading (e.g. "Searching for Mr. Peabody showtimes...").
Register the XML file containing the voice commands at application first launch. The voice commands can be updated after the fact, as well, but like a Background Task, you must "install" the commands when your app is run for the first time.
Handle the voice activation, once Cortana recognizes that the user spoke your app name, followed by a command you've registered". You'll receive, on application activation/page navigation (WinRT app vs Silverlight app) parameters that indicate which voice command was recognized, the fill_in_the_blank_phrases, as well as the full text that was recognized (or typed) into Cortana's UX.
What use cases should developers focus on first?
Applications that help users communicate with people, or the world, Apps that help the user remember or do something later, or applications that help the user find information, or objects, and optionally do something with those things ... All of these application patterns are good application patterns to implement when integrating applications into the Cortana experience. Primarily, that's because these patterns are the most powerful scenarios that help Cortana users the most.
What is the ultimate integration of Cortana you'd like to see?
I'd love for 100+ of the top 500 applications in the store today to integrate with Cortana, especially ones that match the patterns I just described. If I had to pick a single application that I'd personally like to see integrate with Cortana ... I guess I'd say Amazon Mobile, or maybe IMDB. If you would have asked me a couple months ago, I would have said Flixster, but they're already working on that now.
What are you working on now? Does work continue on Cortana or are you jumping to a new, awesome project?
I'm continuing to work inside CU, on the Platform and APIs, working with the same folks I worked with on WP 8.1, trying to take the next big step for NUI for our users. I've been working in or around the NUI space for 15+ of my ~19 years here at Microsoft. We're not done yet, and so ... I'm not done yet either...
[UPDATE] Check out the Q&A with Rob Chambers here: https://channel9.msdn.com/Blogs/Dev-Inspiration/What-Developers-Need-To-Know-About-the-Cortana-API