NUI for PowerPoint for v1

It's been a good number of months since this project was mentioned in the Gallery and so when I saw it was updated for the Kinect for Windows SDK v1...

Kinect PowerPoint Control

This project shows how to use the Kinect SDK to do basic gesture recognition to control PowerPoint.

Usage

1. Compile and run the application
2. Point the Kinect at you and stand at least five feet away
3. You can see yourself in the application window and the three ellipses will track your head and hands.
4. Extend your right arm to activate the "right" or "forward" gesture. Extend your left arm to active the "left" or "back" gesture. These gestures will send a right or left arrow key to the foreground application, respectively.
5. Run your PowerPoint show so PowerPoint is the foreground application, and the right and left gestures will go forward and back in your deck.

The ellipses grow and change color when your hand exceeds the threshold of 45 centimeters. The gestures will only activate once as your hand exceeds the threshold, and only one of the gestures can be active at once. You must bring your hand back closer to your body to activate the gesture a second time.
The gestures will also work for any other application. For example, open Notepad and type some text then use the gestures to move the cursor left or right one character at a time.

Speech recognition

By default voice recognition is disabled due to false positives, but you can uncomment the call to the StartSpeechRecognition() in MainWindow_Loaded() to enable speech recognition. The recognizer is set up to recognize and respond to the following commands:

• computer show window
• computer hide window
• computer show circles
• computer hide circles

Limitations

1. There is currently no way to activate embedded videos, so you should add a PowerPoint animation so the video starts when you push the right arrow key.
2. The gesture is triggered based upon the distance between the head and the hands, so you might accidentally trigger the gesture if you put your arms out, or bend over to pick something up perhaps.
3. Speech recognition, if enabled, will currently cause the program to hang after you close the window until the next audio sample is recognized or rejected.
4. Speech recognition may have a lot of false positive or false negatives. Hopefully this will be improved in future versions of the SDK.

Project Information URL: http://kinectpowerpoint.codeplex.com/

Project Download URL: http://kinectpowerpoint.codeplex.com/releases/view/83141

Project Source URL: http://kinectpowerpoint.codeplex.com/SourceControl/list/changesets

private void ProcessForwardBackGesture(Joint head, Joint rightHand, Joint leftHand)
{
     if (rightHand.Position.X > head.Position.X + 0.45)
     {
         if (!isBackGestureActive && !isForwardGestureActive)
         {
             isForwardGestureActive = true;
             System.Windows.Forms.SendKeys.SendWait("{Right}");
         }
     }
     else
     {
         isForwardGestureActive = false;
     }

     if (leftHand.Position.X < head.Position.X - 0.45)
     {
         if (!isBackGestureActive && !isForwardGestureActive)
         {
             isBackGestureActive = true;
             System.Windows.Forms.SendKeys.SendWait("{Left}");
         }
     }
     else
     {
         isBackGestureActive = false;
     }
}

void SreSpeechRecognized(object sender, SpeechRecognizedEventArgs e)
{
    //This first release of the Kinect language pack doesn't have a reliable confidence model, so 
    //we don't use e.Result.Confidence here.
    if (e.Result.Confidence < 0.70)
    {
        Trace.WriteLine("\nSpeech Rejected filtered, confidence: " + e.Result.Confidence);
        return;
    }

    Trace.WriteLine("\nSpeech Recognized, confidence: " + e.Result.Confidence + ": \t{0}", e.Result.Text);

    if (e.Result.Text == "computer show window")
    {
        this.Dispatcher.BeginInvoke((Action)delegate
            {
                this.Topmost = true;
                this.WindowState = System.Windows.WindowState.Normal;
            });
    }
    else if (e.Result.Text == "computer hide window")
    {
        this.Dispatcher.BeginInvoke((Action)delegate
        {
            this.Topmost = false;
            this.WindowState = System.Windows.WindowState.Minimized;
        });
    }
    else if (e.Result.Text == "computer hide circles")
    {
        this.Dispatcher.BeginInvoke((Action)delegate
        {
            this.HideCircles();
        });
    }
    else if (e.Result.Text == "computer show circles")
    {
        this.Dispatcher.BeginInvoke((Action)delegate
        {
            this.ShowCircles();
        });
    }
}

Contact Information:

• Blog: http://nui.joshland.org/
• Twitter: @joshblake

Page image, Powerpoint Karaoke, courtesy of Mike Licht, NotionsCapital.com