Can the Kinect "know" what you are doing?
Description
One of the more common Kinect demo's we see is someone controlling the video on an XBox, via voice or gesture. That's cool and all, but what about Windows Media Player? How do we get that same kind of cool NUI on our PC's?
Well, you get a Kinect and then...
Windows Media Player NUI
Demonstrating THE how you can use the Microsoft Kinect SDK to control windows media player. The software written in C# can detect when a person moves away from the TV and pauses the video for him/her/it. Upon return you can rewind and play the video again by voice command.
I've been interested in Natural User Interfaces for some time and frequent websites such as Nui Group often. With the release of the Kinect SDK I decided to give making a Media Player NUI a shot and released a demo a few weeks ago. Since then Mark and I made some changes, and thought I'd do an update, show some code, and allow others to download and try it. We intend to make another demo soon showing the new features. This code requires having the Kinect NUI and Audio operational as well as the Windows Media Player SDK.
We started with a Windows Form Application in Microsoft Visual C# 2010 Express and put a media player and a timer on the screen, as well as following all the setup required for the Kinect
...
Project Information URL: http://mathieu-lessard.blogspot.com/2011/07/windows-media-player-nui.html
Project Download URL: http://www.mathieu-lessard.com/wmpnui.zip
Project Source URL: http://www.mathieu-lessard.com/wmpnui.zip
http://www.youtube.com/watch?v=4i6jQh2gmyU
const int STOP_TIMER = 10;
int timer = 10;
void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
//reset watchdog timer that stops the video from playing
timer = STOP_TIMER;
}
private void timer1_Tick(object sender, EventArgs e)
{
timer--;
if (timer < 0)
{
voice_enabled = true;
axWindowsMediaPlayer.Ctlcontrols.pause();
}
}
void AudioThread()
{
source = new KinectAudioSource();
source.FeatureMode = true;
source.AutomaticGainControl = false; //Important to turn this off for speech recognition
source.SystemMode = SystemMode.OptibeamArrayAndAec; //No AEC for this sample
//source.
ri = SpeechRecognitionEngine.InstalledRecognizers().Where(r => r.Id == RecognizerId).FirstOrDefault();
sre = new SpeechRecognitionEngine(ri.Id);
if (ri == null)
{
Console.WriteLine("Could not find speech recognizer: {0}. Please refer to the sample requirements.", RecognizerId);
return;
}
var commands = new Choices();
commands.Add("play");
commands.Add("rewind");
commands.Add("pause");
commands.Add("stop");
commands.Add("fast forward");
commands.Add("volume up");
commands.Add("volume down");
commands.Add("up");
commands.Add("down");
commands.Add("mute");
commands.Add("fullscreen");
commands.Add("maximize");
commands.Add("remote");
//Others to add: Minimize, Min, Max, exit, terminate, enhance, alt f4, I'll be back, keep playing, close remote.
//Minimizing the window is on a different thread and cannot be easily controlled from here. Making minimize just
//switch to non-fullscreen is just silly.
var gb = new GrammarBuilder();
//Specify the culture to match the recognizer in case we are running in a different culture.
gb.Culture = ri.Culture;
gb.Append(commands);
// Create the actual Grammar instance, and then load it into the speech recognizer.
var g = new Grammar(gb);
sre.LoadGrammar(g);
sre.SpeechRecognized += new EventHandler<SpeechRecognizedEventArgs>(SreSpeechRecognized);
s = source.Start();
sre.SetInputToAudioStream(s,
new SpeechAudioFormatInfo(
EncodingFormat.Pcm, 16000, 16, 1,
32000, 2, null));
sre.RecognizeAsync(RecognizeMode.Multiple);
}
Contact Information:
- Blog: [Blog URL]
- Twitter: [Twitter]
Today's project was found in the Kinect forums, Windows Media Player NUI using Kinect
