Tutorial: XNA 4.0 + Kinect for Windows SDK

We don't have many XNA 4 Kinect examples or tutorials in the Gallery, so as soon as I came across this I posted it. I just wish I'd seen it sooner (it was written in June 2011). Better late than never I guess?

Tutorial: XNA 4.0 + Kinect for Windows SDK

...a new tutorial series that will cover the basics of the new Kinect for Windows SDK using XNA 4.0.

Kinect Fundamentals #1: Installation & Setup

Connecting the Kinect

Step 0:
If you don’t have a Kinect Sensor, you can (probably) find it at your nearest electronics dealer.

Step 1:
Download and install the Kinect SDK:

...

Kinect Fundamentals #2: Basic programming

Our first Kinect program will be using XNA 4.0 to create a texture that is updated with a new image from the Kinect Sensor every time a new image is created, thus displaying a video.

Setting up the project
Create a new XNA Windows game and give it a name. To use the Kinect SDK, you will need to add a reference to it. This can be done by right clicking on the References folder, click Add Reference.. and in the .NET tab find Microsoft.Research.Kinect.

...

Creating the NUI object
Now we are ready to create the object that will “hold” the Kinect Sensor. The Kinect SDK have a class named Runtime that contains the NUI library. To get what you need our from the Kinect Sensor, instantiate an object from this class:
Runtime kinectSensor;

We also create a Texture2D object that will contain our images

...

Kinect Fundamentals #3: Getting distance-data from the Depth Sensor

Now that you know how to use the RGB Camera data, it’s time to take a look at how you can use the depth data from the Kinect sensor.

It’s quite similar to getting data from the RGB image, but instead of RGB values, you have distance data. We will convert the distance into a black and white image representing the depth map. Remember, there are two sensors that contains distance data, so this need to be handled.

...

Converting the depth data

Now is the time for the meat of this tutorial. Here we get the depth data from the device in millimeter, and convert it into a distance we can use for displaying a black and white map of the depth. The Kinect device got a range from 0.85m to 4m. We can use this knowledge to create a black and white image where each pixel is the distance from the camera. White pixels are close, while black are far. We might also get some unknown depth pixels if the rays are hitting a window, shadow, mirror and so on (these will have the distance of 0).

Because we are using a Depth Image, there are two bytes per pixel that represents the distance (one from each depth sensor). To get the distance at a given pixel, you will need to bitshift the second byte left by 8.

Project Information URL: http://forums.create.msdn.com/forums/t/85214.aspx ->
1 - Installation and setup (http://digitalerr0r.wordpress.com/2011/06/20/kinect-fundamentals-1-installation-setup/)
2 - Basic programming (http://digitalerr0r.wordpress.com/2011/06/20/kinect-fundamentals-2-basic-programming/)
3 - Getting distance-data from the Depth Sensor (http://digitalerr0r.wordpress.com/2011/06/21/kinect-fundamentals-3-getting-data-from-the-depth-sensor/)

Project Source URL: http://digitalerr0r.darkcodex.net/Tutorials/KinectFundamentals_Tutorial1.rar, http://digitalerr0r.darkcodex.net/Tutorials/KinectFundamentals_tutorial3.rar

void kinectSensor_DepthFrameReady(object sender, ImageFrameReadyEventArgs e)
{
    PlanarImage p = e.ImageFrame.Image;

    Color[] DepthColor = new Color[p.Height * p.Width];

    float maxDist = 4000;
    float minDist = 850;
    float distOffset = maxDist – minDist;

    kinectDepthVideo = new Texture2D(GraphicsDevice, p.Width, p.Height);

    int index = 0;
    for (int y = 0; y < p.Height; y++)
    {
       for (int x = 0; x < p.Width; x++, index += 2)
       {
          int n = (y * p.Width + x) * 2;
          int distance = (p.Bits[n + 0]  |  p.Bits[n + 1] << 8);
          byte intensity = (byte)(255 – (255 * Math.Max(distance – minDist, 0) / (distOffset)));
          DepthColor[y * p.Width + x] = new Color(intensity, intensity, intensity);

        }
    }
    kinectDepthVideo.SetData(DepthColor);
}

protected override void Draw(GameTime gameTime)
{
    GraphicsDevice.Clear(Color.CornflowerBlue);
    
    // TODO: Add your drawing code her

    spriteBatch.Begin();
    spriteBatch.Draw(kinectDepthVideo, new Rectangle(0, 0, 640, 480), Color.White);
    spriteBatch.Draw(overlay, new Rectangle(0, 0, 640, 480), Color.White);
    spriteBatch.End();

    base.Draw(gameTime);
}

Contact Information:

• Blog: http://digitalerr0r.wordpress.com/
• Twitter: @petriw