Adventures in Creating a Cortana Speech Helper Library
- Posted: May 13, 2015 at 6:00AM
Rob Miles, Friend of the Blog and Microsoft MVP, recently blogged about something you all know I really dig, SI (Speech Interface), i.e. Cortana. Best of all he's created a helper that will speed up the writing of your own speech enabled applications...
Using Speech in a Windows Phone application is quite easy. But it could be easier. You have to create the grammar file and then hook it up to your program code. Being an inventive soul I've made a little helper library. This has been created for use in an adventure which uses traditional two word commands:Go west
The helper library lets you create as many verb-noun combinations as you like, and automatically builds and installs the Voice Commands file for you. You can use it to build an adventure, but you could also use it in any situation where you want to implement simple command processing using voice input.
The best way to get started is to load up the AdventureDemo project and take a look at how it works. It contains the AdventureEngine class along with a sample adventure which does very little. The project also contains customised code in App.xaml.cs which sets up the speech command file and decodes and despatches spoken commands. There is also code in MainPage.xaml.cs that performs the voice commands. You can just copy these files into your project.
Creating your own Adventure
The GameDesign class is the base class for your adventures. It serves as the container for the adventure commands and provides the methods that will generate the XML voice command files. It also provides the basis of adventure save and load behaviours. As supplied it just stores the text of all the previous commands and responses but you can extend the load and save pattern to include saving the position of the player and the state of game objects.
Creating an Adventure class of your own
You create your own adventure by extending the GameDesign class:
Setting up Adventure Commands
The next thing you need to do is set up the commands that you want the adventure to use. There are two kinds of commands, one is a single command with no options, for example "Look", and the other command is followed by an option, for example "Go West". In the sample code above you can see how I've set these two commands up.
Creating the Voice Command file
The VoiceCommands list in the GameDesign class holds all the commands that have been created. These are automatically stored in a file and then loaded as voice commands when the program starts. You don't have to do anything else. The GameDesign class provides a method that will generate a voice command file for a number of language options. The voice command file is actually generated in the App.xaml.cs file when an application is launched. Take a look for the method setupVoiceCommands if you are interested in how it all works. The language locale for the voice commands is set to the current language locale for the phone, so the commands should always work correctly.
Decoding the Voice commands
The spoken commands are decoded in the App.xaml.cs file. This also automatically determines the destination page to be activated when the program is activated by a voice command.
Decoding the Commands
The GameDesign class provides a method that will decode and act on game commands issued by the player. The method is used in the OnNavigatedTo method for the MainPage in the sample application. The method loads the game status, performs the command and then saves the game status. It is important that your game works in this way, since you can't make any assumptions about when the game might be in a position to save data.
If the program was started from a voice command it will also speak the response for the game. The GameDesign class can also decode commands which are typed in from the keyboard. When the user types commands it is possible that they might give invalid ones, and so your game needs to provide a method that can deliver a sensible response in this situation.
- Get the code
- Universal Windows App Development with Cortana and the Speech SDK (free course)
- Windows/Windows Phone 8.1 SDK