Body Tracking with the Kinect for Windows v2
It's Kinect for Windows v2 Thursday, with another great piece of work from Vangos Pterneas. Today he continues to show us the Body Tracking features and capabilities found in the Kinect for Windows v2 device.
Some of the other times we've highlighted Vangos Pterneas's work;
- Looking at the Kinect for Windows v2
- Color, depth and infrared streams in the Kinect for Windows v2 world (here's how)
- Kinect gestures, an implementation walk through
- Put down the measuring tape. Using the Kinect to find your height
NOTE: This is preliminary software and/or hardware and APIs are preliminary and subject to change.
In my previous blog post, I show you how to display the color, depth and infrared streams of Kinect version 2 by transforming the raw binary data into Windows bitmaps.
This time, we’ll dive into the most essential part of Kinect: Body tracking.
The initial version of Kinect allowed us to track up to 21 body joints. The second version allows up to 25 joints. The new joints include the fists and thumbs! Moreover, due to the enhanced depth sensor, the tracking accuracy has been significantly improved. Experienced users will notice less jittering and much better stability. Once again, I would like to remind you of my video, which demonstrates the new body tracking capabilities:
Extending the project
In the previous blog post, we created a project with an <Image> element for displaying the streams. We now need to add a <Canvas> control for drawing the body. Here is the updated XAML code:
The Reader_MultiSourceFrameArrived method will be called whenever a new frame is available. Let’s specify what will happen in terms of the body data:
- Get a reference to the body frame
- Check whether the body frame is null – this is crucial
- Initialize the _bodies list
- Call the GetAndRefreshBodyData method, so as to copy the body data into the list
- Loop through the list of bodies and do awesome stuff!
Always remember to check for null values. Kinect provides you with approximately 30 frames per second – anything could be null or missing! Here is the code so far:
This is it! We now have access to the bodies Kinect identifies. Next step is to display the skeleton information on-screen. Each body consists of 25 joints. The sensor provides us with the position (X, Y, Z) and the rotation information for each one of them. Moreover, Kinect lets us know whether the joints are tracked, hypothsized or not tracked. It’s a good practice to check whether a body is tracked before performing any critical functions. The following code illustrates how we can access the different body joints:
The supported joints by Kinect 2 are the following:
Neck and thumbs are new joints added in the second version of Kinect.
Knowing the coordinates of every joint, we can now draw some objects using XAML and C#. However, Kinect provides a distance in millimetres, so we need to map the millimetres to screen pixels. In the attached project, I have made this mapping for you. So, the only method you need to call is the DrawPoint or the DrawLine. Here is the DrawPoint:
Project Information URL: http://www.codeproject.com/Articles/743862/Kinect-for-Windows-version-Body-tracking