Alarm Clock with voice Commands Sample

This sample demonstrates how to manage an alarm by using voice commands.

The sample demonstrates the basics of the voice commands feature, which is a part of the speech feature for Windows Phone 8. The sample also demonstrates how to use the Alarm class. After the app launches, exit the app to the Start screen and tap and hold the Start button to view and accept the speech privacy policy. You can then use voice commands to set an alarm, turn off the alarm, and so on. A full list of the available voice commands are in the voicecommands.xml file, which is located in the root folder of the app.

This sample uses the Windows.Phone.Speech.VoiceCommands Windows Phone Runtime API. For more info about the speech feature in Windows Phone 8, see Speech for Windows Phone 8.

When you grab the sample and play with it in the Emulator, you long press/click the Start button (the Windows logo on the bottom of the phone, in the middle), accept the speech recognition warning, say the application, "Alarm Clock" and then the commands, "Turn alarm on", "Turn alarm off" or "Set alarm for [time]". Being the guy I am, I wasn't saying the application first. No wonder the phone had no clue what I was actually trying to tell it what to do...

Not having done any speech recognition app's for Windows Phone 8 before (or any Windows Phone version for that matter), what I really liked was how all the voice commands being listened for were all in one place, in an easily readable layout, in the voicecommands.xml

The app load's, loads the XML and then setups up an async listening command.

When a command comes in, it branches based on the command and also shows how easy it is to respond back with speech.

In short, this shows off just how easy it is to add speech recognition and voice to your app's...

Speech Recognition for the Kinect, the Easy Way...

Speech Recognition for the Kinect:

In the development of Kin-educate I found this to be one of the most tricky parts. Largely because I couldn't find any complete tutorials out there other than the quick start series at channel 9, which, if you have checked it out, you will know is helpful, but not comprehensive.

I have been asked by quite a lot of people about how I did the speech recognition in the maths game for Kin-educate, so I thought I would do a quick tutorial that cuts out all the unnecessary bits, and just focuses on getting you set up and speech recognition working quickly and easily. This tutorial assumes you have a Kinect project set up already - if you do, you should be able to just copy and paste this code, in order, and you're all set!

...

Project Information URL: http://kin-educate.blogspot.com/2012/06/speech-recognition-for-kinect-easy-way.html

Contact Information:

• Blog: http://kin-educate.blogspot.com

Context-Aware Dialogue with Kinect

Meet Lily, my office assistant. We converse often, and at my direction Lily performs common business tasks such as looking up information and working with Microsoft Office documents. But more important, Lily is a virtual office assistant, a Microsoft Kinect-enabled Windows Presentation Foundation (WPF) application that’s part of a project to advance the means of context-aware dialogue and multimodal communication.

Before I get into the nuts-and-bolts code of my app—which I developed as part of my graduate work at George Mason University—I’ll explain what I mean by context-aware dialogue and multimodal communication.

Context-Aware Dialogue and Multimodal Communication

As human beings, we have rich and complex means of communicating. Consider the following scenario: A baby begins crying. When the infant notices his mother is looking, he points at a cookie lying on the floor. The mother smiles in that sympathetic way mothers have, bends over, picks up the cookie and returns it to the baby. Delighted at the return of the treasure, the baby squeals and gives a quick clap of its hands before greedily grabbing the cookie.

This scene describes a simple sequence of events. But take a closer look. Examine the modes of communication that took place. Consider implementing a software system where either the baby or the mother is removed and the communication is facilitated by the system. You can quickly realize just how complex and complicated the communication methods employed by the two actors really are. There’s audio processing in understanding the baby’s cry, squeal of joy and the sound of the clap of hands. There’s the visual analysis required to comprehend the gestures repre­sented by the baby pointing at the cookie, as well as inferring the mild reproach of the mother by giving the sympathetic smile. As often is the case with actions as ubiquitous as these, we take for granted the level of sophistication employed until we have to implement that same level of experience through a machine.

Let’s add a little complexity to the methods of communication. Consider the following scenario. You walk into a room where several people are in the middle of a conversation. You hear a single word: “cool.” The others in the room look to you to contribute. What could you offer? Cool can mean a great many things. For example, the person might have been discussing the temperature of the room. The speaker might have been exhibiting approval of something (“that car is cool”). The person could have been discussing the relations between countries (“negotiations are beginning to cool”). Without the benefit of the context surrounding that single word, one stands little chance of understanding the meaning of the word at the point that it’s uttered. There has to be some level of semantic understanding in order to comprehend the intended meaning. This concept is at the core of this article.

Project Lily

I created Project Lily as the final project for CS895: Software for Context-Aware Multiuser Systems at George Mason University, taught by Dr. João Pedro Sousa. As mentioned, Lily is a virtual assistant placed in a typical business office setting. I used the Kinect device and the Kinect for Windows SDK beta 2. Kinect provides a color camera, a depth-sensing camera, an array of four microphones and a convenient API that can be used to create natural UIs. Also, the Microsoft Kinect for Windows site (microsoft.com/en-us/kinectforwindows) and Channel 9 (bit.ly/zD15UR) provide a plethora of useful, related examples. Kinect has brought incredible capabilities to developers in a (relatively) inexpensive package. This is demonstrated by Kinect breaking the Guinness World Records “fastest selling consumer device” record (on.mash.to/hVbZOA). The Kinect technical specifications (documented at bit.ly/zZ1PN7) include:

Project Information URL: http://msdn.microsoft.com/en-us/magazine/hh882450.aspx

Project Source URL: http://archive.msdn.microsoft.com/mag201204Kinect

• Take advantage of sensors like accelerometers, light sensors, compasses, GPS etc.
• Access content from connected portable devices like music players and digital cameras.
• Capture and create video from webcams and screen output
• Use speech recognition and text to speech capabilities.
• Integrate with the Windows 7 taskbar (Jump Lists, Icon Overlays, Taskbar Progress etc.)
• And more!

Head over to http://code.msdn.microsoft.com/nesl to start developing with it.

Here is a bit of his speech recognition code using the System.Speech.dll:

let recognizerEvent = getWordRecognizer()

// Main handler - convert spoken text into RL commands
let handleWord spokenText =
    printfn "Recognized Word: %s" spokenText
    let action = 
        match spokenText with
        | "up"  // Has a hard time recognizing this :(
        | "north" -> MoveUp(20)
        | "down"  -> MoveDown(20)
        | "left"  -> MoveLeft(20)
        | "right" -> MoveRight(20)

        | "fire"  -> Fire

        | _ -> NoOp
        
    performAction rocketLauncher action |> ignore

// Exit handler - specifically look for exit/quit
let terminateLoop = ref false
let terminateLoopHandler = function | "exit" 
                                    | "quit" -> terminateLoop := true
                                    | _      -> ()

// Hook up event handlers
recognizerEvent.Add(handleWord)
recognizerEvent.Add(terminateLoopHandler)

while terminateLoop.Value = false do
    System.Threading.Thread.Yield() |> ignore
()

Chris happily provided the source code as well, RocketLauncher_v1.0.zip. 

Om Malik has the relevant details for those who are interested in the plumbing of the deal. According to him, Mr. Ballmer was personally driving the deal and TellMe will continue as an independent subsidiary of Microsoft.

Perception of the purchase is that TellMe’s technology will help differentiate Microsoft in the search business, particularly mobile search (actually speaking to your mobile phone—surely you’re joking). Don Dodge points out in his post about the deal that both mobile and local search are very valuable (and relatively open) markets.

While the public perception seems to revolve around Microsoft’s attempts to become more relevant in the search market, Richard Sprague believes the deal is about acquiring TellMe’s data: “It's extremely important in speech to get actual users using your system, because every call and every interaction is new data that will improve quality and accuracy. A significant fraction of the U.S. population has called TellMe, which is an asset that will mean huge improvements to all of our systems.”