Open source Kinect gesture recognition project, Kinect DTW

Description

Seems like we're on a Gesture recording theme this week so far... Today's project provides another look at recognizing and recording gestures with the Kinect for Windows SDK. In today's we get the source to the main library too, so we can see just how the magic is done...

Kinect SDK Dynamic Time Warping (DTW) Gesture Recognition

This project allows developers to include fast, reliable and highly customizable gesture recognition in Microsoft Kinect SDK C# projects. It uses skeletal tracking and currently supports 2D vectors.

Included is a gesture recorder, recognizer and sample gestures. You can save your gestures to file and we'd love it if you shared your gestures with the community - share and share alike!

This project is currently in setup mode and only available to project coordinators and developers. Once you have finished setting up your project you can publish it to make it available to all CodePlex visitors.

I would really love it if you share the gestures you make, as well as providing feedback on the best parameter settings you have found. I'm also keen for others to improve the project, so please feel free to contribute.

You can use this code or any original or modified portion of it in any project that you like. We only ask that you keep the authors' names in the copyright notice because a lot of thought and effort went into this.

For additional information, please check out

KinectDTW - open source Kinect gesture recognition project released

This evening I published to Codeplex the gesture recording and recognition project I’ve been working on. Massive shout out to Rhemyst for doing most of the thinking on this. Grab it here:

http://kinectdtw.codeplex.com/

Key features:

  • Gesture recording
  • Gesture recognition
  • Save gestures to file
  • Tweakable parameters for fine control of how the recognizer works
  • Works out of the box - just run the sample app
  • Source code available too if you want to make your own gestures

Please try this out and if you find it useful I'd love you to contribute to the project. There are a hundred things I can think of that could be done better but this is a great starting point for anyone who is interested in vector-based recognition with the Kinect.

Project Information URL: http://kinectdtw.codeplex.com/

Project Download URL: http://kinectdtw.codeplex.com/releases/view/70946

Project Source URL: http://kinectdtw.codeplex.com/SourceControl/list/changesets

kinectdtwkinectdtw2

/// <summary>
/// Recognize gesture in the given sequence.
/// It will always assume that the gesture ends on the last observation of that sequence.
/// If the distance between the last observations of each sequence is too great, or if the overall DTW distance between the two sequence is too great, no gesture will be recognized.
/// </summary>
/// <param name="seq">The sequence to recognise</param>
/// <returns>The recognised gesture name</returns>
public string Recognize(ArrayList seq)
{
     double minDist = double.PositiveInfinity;
     string classification = "__UNKNOWN";
     for (int i = 0; i < _sequences.Count; i++)
     {
         var example = (ArrayList) _sequences[i];
         ////Debug.WriteLine(Dist2((double[]) seq[seq.Count - 1], (double[]) example[example.Count - 1]));
         if (Dist2((double[]) seq[seq.Count - 1], (double[]) example[example.Count - 1]) < _firstThreshold)
         {
             double d = Dtw(seq, example) / example.Count;
             if (d < minDist)
             {
                 minDist = d;
                 classification = (string)_labels[i];
             }
         }
     }

     return (minDist < _globalThreshold ? classification : "__UNKNOWN") + " " /*+minDist.ToString()*/;
}

 

Contact Information: