Templated posture detector, Voice Commander and NuGet package now available in v1.1 of the [David Catuhe] Kinect Toolkit

We're revisiting a friend of the Gallery, David Catuhe (aka Eternal Coding), and his Kinect Toolkit. We last covered the Kinect Toolkit here, Gestures and Tools for Kinect and matching Toolkit too (and don't forget he was our first Gallery post too, Of course our first Kinect for Windows SDK Project has to involve a Light Saber!)

Anyway... David has continued to enhance and improve his Kinect Toolkit and recently released v1.1 adding some cool new features and making it easier to add to our projects too.

Kinect Toolbox 1.1 : Template based posture detector and Voice Commander

Kinect Toolbox v1.1 is now out and this new version adds support for some cool features:

  • Templated posture detector
  • Voice Commander
  • NuGet package

You can find the toolbox here : http://kinecttoolbox.codeplex.com or you can grad it using NuGet : http://nuget.org/List/Packages/KinectToolbox

Templated posture detector

Using the same algorithm as TemplatedGestureDetector, you can now use a learning machine and a matching system to detect postures. In the sample attached with the toolbox I detect the ”T” posture (i.e. when you body is like the T letter):

...

Voice Commander

One thing worth noting when you develop with Kinect is that you will spend your time getting up and sitting down Sourire. In the previous article, I introduced the replay system which is very useful to record a Kinect session.

But when you are alone, even the recording is painful because you cannot be at the same time in front of the sensor and in front of your keyboard to start/stop the record.

So here enters the Voice Commander (tadam!!). This class can use a list of words and raise an event when it detect one of them (using the microphone array of the sensor). So for example, you can use “record” and “stop” orders to launch and stop the recording session while you stay in front of the sensor!

The code is really simple (thanks to Kinect for Windows SDK and Microsoft Speech Platform SDK):

...

Project Information URL: http://blogs.msdn.com/b/eternalcoding/archive/2011/08/02/kinect-toolkit-1-1-template-based-posture-detector-and-voice-commander.aspx, http://kinecttoolbox.codeplex.com

Project Download URL: http://kinecttoolbox.codeplex.com/releases/view/71078, http://nuget.org/List/Packages/KinectToolbox

Project Source URL: http://kinecttoolbox.codeplex.com/SourceControl/list/changesets

image

image

public class VoiceCommander
{
     const string RecognizerId = "SR_MS_en-US_Kinect_10.0";
     Thread workingThread;
     readonly Choices choices;
     bool isRunning;

    public event Action<string> OrderDetected;

    public VoiceCommander(params string[] orders)
     {
         choices = new Choices();
         choices.Add(orders);
     }

    public void Start()
     {
         workingThread = new Thread(Record);
         workingThread.IsBackground = true;
         workingThread.SetApartmentState(ApartmentState.MTA);
         workingThread.Start();  
     }

    void Record()
     {
         using (KinectAudioSource source = new KinectAudioSource
         {
             FeatureMode = true,
             AutomaticGainControl = false,
             SystemMode = SystemMode.OptibeamArrayOnly
         })
         {
             RecognizerInfo recognizerInfo = SpeechRecognitionEngine.InstalledRecognizers().Where(r => r.Id == RecognizerId).FirstOrDefault();

            if (recognizerInfo == null)
                 return;

            SpeechRecognitionEngine speechRecognitionEngine = new SpeechRecognitionEngine(recognizerInfo.Id);

            var gb = new GrammarBuilder {Culture = recognizerInfo.Culture};
             gb.Append(choices);

            var grammar = new Grammar(gb);

            speechRecognitionEngine.LoadGrammar(grammar);
             using (Stream sourceStream = source.Start())
             {
                 speechRecognitionEngine.SetInputToAudioStream(sourceStream, new SpeechAudioFormatInfo(EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, null));

                isRunning = true;
                 while (isRunning)
                 {
                     RecognitionResult result = speechRecognitionEngine.Recognize();

                    if (result != null && OrderDetected != null && result.Confidence > 0.7)
                         OrderDetected(result.Text);
                 }
             }
         }
     }

    public void Stop()
     {
         isRunning = false;
     }
}

Here's an example of using the Voice Commander in your code;

voiceCommander = new VoiceCommander("record", "stop");
voiceCommander.OrderDetected += voiceCommander_OrderDetected;

voiceCommander.Start();

void voiceCommander_OrderDetected(string order)
{
     Dispatcher.Invoke(new Action(() =>
     {
         if (audioControl.IsChecked == false)
             return;

        switch (order)
         {
             case "record":
                 DirectRecord(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "kinectRecord" + Guid.NewGuid() + ".replay"));
                 break;
             case "stop":
                 StopRecord();
                 break;
         }
     }));
}

Contact Information:

Follow the Discussion

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.