Understanding the Kinect Coordinate Mapping, with a little help from Vangos Pterneas

Vangos Pterneas, Friend of the Gallery, has posted another great article, this time helping all of us understand the Kinect Coordinate Mapping for v1 AND v2...

Other recent Gallery Posts for Vangos Pterneas;

Understanding Kinect Coordinate Mapping

This is another post I publish after getting some good feedback from my blog subscribers. Seems that a lot of people have a problem in common when creating Kinect projects: how they can properly project data on top of the color and depth streams.

As you probably know, Kinect integrates a few sensors into a single device:

  • An RGB color camera – 640×480 in version 1, 1920×1080 in version 2
  • A depth sensor – 320×240 in v1, 512×424 in v2
  • An infrared sensor – 512×424 in v2

These sensors have different resolutions and are not perfectly aligned, so their view areas differ. It is obvious, for example, that the RGB camera covers a wider area than the depth and infrared cameras. Moreover, elements visible from one camera may not be visible from the others. Here’s how the same area can be viewed by the different sensors:

An example

Suppose we want to project the human body joints on top of the color image. Body tracking is performed using the depth sensor, so the coordinates (X, Y, Z) of the body points are correctly aligned with the depth frame only. If you try to project the same body joint coordinates on top of the color frame, you’ll find out that the skeleton is totally out of place:

image

CoordinateMapper

Of course, Microsoft is aware of this, so the SDK comes with a handy utility, named CoordinateMapper. CoordinateMapper’s job is to identify whether a point from the 3D space corresponds to a point in the color or depth 2D space – and vice-versa. CoordinateMapper is a property of the KinectSensor class, so it is tight to each Kinect sensor instance.

You can download a test project from GitHub and check how CoordinateMapper is used. To understand it more thoroughly, continue reading this tutorial.

...

[Please click through for the rest of the article, code and information]

Project Information URL: http://pterneas.com/2014/05/06/understanding-kinect-coordinate-mapping/

Project Source URL: Kinect for Windows version 1, SDK 1.8, Kinect for Windows version 2, SDK 2.0

Contact Information:

Tags:

Follow the Discussion

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.