Working with Depth Data

Download this episode

Download Video

Description

In the working with depth data Quickstart video, we'll cover:

  • An overview of how depth data represents the distance and player for every pixel
  • An overview of Default and Near Mode and what features are supported in both
  • An explanation of what distance data the Kinect returns including knowing when you are too near, too far, or the distance is unknown
  • The formulas for calculating depth and player information
  • How to visualize depth and player data, including a monochrome histogram and player colorization
  • How to use the WPF DepthViewer control to understand depth data returned from the Kinect

Resources

Embed

Format

Available formats for this video:

Actual format may change based on video formats available and browser capability.

    The Discussion

    • User profile image
      El Bruno

      Great demo ... Hope to get soon the Kinect for Windows sensor in Spain.

      Regards Big Smile

    • User profile image
      Yakko Warner

      Is anyone else reading the description to this video in [Sesame Street's] Grover's voice?

      http://www.youtube.com/watch?v=iZhEcRrMA-M

    • User profile image
      Adam​Speight2008

      What happens at 2000mm? Dan  Wink  Undefined behavior ?

    • User profile image
      SergioPassos

      Great video, it's so good to be able to clearly understand what your saying.

      I just want to suggest you use some sort of sound equalizer to get rid of those extremely high pitched words (your mic is too sensitive).

    • User profile image
      Alina

      Why is there this contour (blue in the 11:00 - 11:05) around the silhuette? Is it an error? And why the hand is doubled in (13:00-13:10)? Is it an effect of being to close to the camera?

    • User profile image
      Dan

      @SergioPassos: Thanks for your feedback, we've add a compressor that ought to keep the high sound, not so high

    • User profile image
      Dan

      @AdamSpeight2008:Good catch! I think I should fix that Smiley

    • User profile image
      Dan

      @Alina: Good question, the Kinect is telling us that the depth is "unknown" for the shadow around the hand. You can see this a bit better around 17:30 on the right image where we discuss how the colorization uses a different color for an "unknown" color.

      Since the depth value is -1, our simple Blue/Green/Red logic decides to color it Blue because of this line:

      if (depth <= 900)

      We could change it and choose a different color to color the unknown distance by checking if the depth is equal to -1 like this:  

      if (depth == -1)

    • User profile image
      Pavel

      Hello Dan,
      What is the MAX limit for players in Kinect? I mean how much players I may use for my needs?
      --
      Thanks.

    • User profile image
      Dan

      @Pavel: For depth data, you will get 6 player proposals back per Kinect. For Skeletal tracking, meaning the position of joints (hand, head, foot, etc) you can only track two skeletons at a time for any PC. This means that if you have multiple Kinects plugged into one machine, you can only be tracking skeletons on one of them. Hope this helps

    • User profile image
      jnsjoh011

      If I want to modify this code to create a point cloud from the depth data, how will I approach that??

    • User profile image
      teoaf

      How do i do to get the the depth real information, i mean the coordinates in the real world not only of Z, but also of X and Y?

    • User profile image
      weekswyj

      Excellent demo. It's so good to understand what your saying so clearly.

    • User profile image
      Colla

      Thanks a lot for your helpful tutorials... i am new to kinect and i was trying to apply the tutorial and it worked perfect,,, your tutorials are really helpful.. Thanks a lot

    • User profile image
      Colla

      I have this problem is when i color the players with different colors "each player with a color",, the program starts to freeze a lot and be very very slow...

      Any ideas?? what can i do ?

      Thanks a lot.

    • User profile image
      Dan

      , teoaf wrote

      How do i do to get the the depth real information, i mean the coordinates in the real world not only of Z, but also of X and Y?

      You may want to look at getting a SkeletalPoint in the skeletal tracking demo as that includes X,Y,Z distances from the Kinect. It's not for every point, but rather for tracked joints (head, elbow, etc)

    • User profile image
      Dan

      @Colla: Try switching to a WriteableBitmap that you set the BitmapSource.Source property to once, and then clear/re-write the bits. If you look at the source code for the WPF Viewers, you'll see an example of how to use the WriteableBitmap.

      Cheers,

      -Dan

    • User profile image
      Max

      Dan, great videos, thank you.

      I've gotten the monochrome coloring to work.
      But for coloring the players gold, when the Kienct recognizes a player, all the frames would freeze. And once the player leaves the frame, the Kinct would resume to color the frame as monochrome. Any thoughts on why?

    • User profile image
      KinectFJ

      Hello together,

      I try to programm the Depth likein this clip , but don't work.

      can some body give me the code for!!

      please help

       

      best regards

    • User profile image
      Dan

      @Max: It's hard to tell without looking at your source code, are you using the built-in quickstart sample?

    • User profile image
      Dan

      @KinectFJ: Are you using the included sample? Does that work? If not, does the Kinect Explorer sample work?

    • User profile image
      KinectFJamal

      @ Dan, it works already;-) thanks for the quick reply...but another question...I have to make a project in fact recognize object like bottle or wall..How should I start at the beginning!!

       

      best regards

       

       

       

    • User profile image
      Dan

      @KinectFJamal: That's a manual process that involves interpreting the data from the RGB and/or depth camera. One library for interpreting objects is Open CV (Computer Vision), here's an example of using it on CodeProject - http://www.codeproject.com/Articles/28465/Easy-to-use-Wrapper-DLL-for-Intel-s-OpenCV-Library

    • User profile image
      Nate

      @Max, I had the same problem. I found I could resolve it by removing the references to Colors.Gold and replacing them with raw numbers. I'm not sure why this fixes the problem, though, since Colors.Gold works fine in the quickstart sample.

    • User profile image
      Max

      @Dan, @Nate,

      I fixed my problem (took a while...)
      Basically, it turns out I was not setting the correct resolution size when initializing, causing it to freeze. The depth, skeleton, and image frames need to have the same size.

    • User profile image
      Dan

      @Max: Thanks for replying, this may be causing issues with other folks too so I'm glad you posted it here!

    • User profile image
      Todd

      The video explains how to obtain depth measurements from the depthFrame, but doesn't say what vector these measurements are along or how to convert them to world XYZ coordinates!

      I looked but I see no function in the SDK for finding the XYZ values. I suppose if one knew the FOV of the depth camera an approximate function could be written. Ideal would be SDK call that checks the depth camera's firmware calibration and gives proper values.

      Pointclouds are probably one of the most popular applications of the Kinect (for visuals anyway), but without this info most people are going to end up doing it the wrong way.

    • User profile image
      Fabricio

      Dan, how can I put the depth, color and skeleton view in the same interface?

    • User profile image
      Steppy

      Hi all,
      is there any way to record or store the depth video (near mode) in a common video file (e.g. .avi, .mpeg)? Thanks.

    • User profile image
      Maximilian30

      In a nutshell, I have the Kinect picking up my motion and I have created a nice skeleton so that I can easily see my forward/backward/side to side movements.  I want to take these forward/backward/side to side movements and translate it to up and down motion on the screen.In other words, say I have the left foot moving forward and backward from the camera, I want to translate that as a dot moving up and down respectively. Is there a way to simple transpose from YZ plane to XY plane?

    • User profile image
      Dan

       

      Dan, how can I put the depth, color and skeleton view in the same interface?

      [/quote]

       

      Check out the Kinect Explorer sample that ships with the Kinect Developer Toolkit as it includes all of the WPF viewers on one Window

    • User profile image
      Dan

      The video explains how to obtain depth measurements from the depthFrame, but doesn't say what vector these measurements are along or how to convert them to world XYZ coordinates!

      I looked but I see no function in the SDK for finding the XYZ values. I suppose if one knew the FOV of the depth camera an approximate function could be written. Ideal would be SDK call that checks the depth camera's firmware calibration and gives proper values.

      Pointclouds are probably one of the most popular applications of the Kinect (for visuals anyway), but without this info most people are going to end up doing it the wrong way.

       

      You can use the KinectSensor class MapDepthToSkeletonPoint method to convert a depth point to a SkeletonPoint.

      A SkeletonPoint should be what you need given that it is meters from the camera

       

    • User profile image
      Juan Pablo

      Hi @Dan I just solved the "freezing frame" problem, it's a nonesense cause when using the values of the Color class (Color.Red.G, ...B, ...R, for example), the program slows down just like @Colla said, but I just changed the values to numbers and it works perfectly for every player. No need to have same frame sizes as @Max said. I also changed the BitmapSource to a WriteableBitmap, and it's working perfectly.

      It worked perfectly for me, but everyone should trySaludos!

    • User profile image
      juanpibanez

      @CollaI just solved the "freezing frame" problem, it's a nonesense cause when using the values of the Color class (Color.Red.G, ...B, ...R, for example), the program slows down just like you said, but I just changed the values to numbers and it works perfectly for every player. I also changed the BitmapSource to a WriteableBitmap, and it's working perfectly.

      Try it.

    • User profile image
      Saud

      Hey Dan,
      I working on this tutorial and so far I sailed through the camera fundamentals, but im having some issues with working with Depth data tutorial. For some reason, im getting this error three times, "The name 'kinectSensorChooser1' does not exist in the current context"

      Could you please explain a little about this error message and how I should go about solving it?
      Thanks

    • User profile image
      nur

      Hello Dan and all other Kinect developers to be,

      I love these tutorial videos that you prepared and have been getting a lot of help from them. However, I have not been able to get the sample code working for this specific part. All of the codes up to this one worked but here all I get is a blank main window.

      Has anyone come across the same issue or knows why I am having this problem?

      Edit: It does not give any error messages.

    • User profile image
      nur

      , nur wrote

      Hello Dan and all other Kinect developers to be,

      I love these tutorial videos that you prepared and have been getting a lot of help from them. However, I have not been able to get the sample code working for this specific part. All of the codes up to this one worked but here all I get is a blank main window.

      Has anyone come across the same issue or knows why I am having this problem?

      Edit: It does not give any error messages.

       

      Got it working! Smiley Thanks!! Smiley

    • User profile image
      Hadi

      i've try this code and improve with my own code.
      its work perfectly. :)

      i want to get the distance data of an object in range 800mm and 2000mm. how ican get it?

      i hve try to display 'depth' with this code:
      textBox1.Text = "Detecting objects in the range " + depth + " mm";
      but its not work...

      help me to do it..,Thanks!

    • User profile image
      Francesco

      Hi all. I have a problem.
      I use a Depth Image in my project but when I use MapFromSkeletonPoint this generates an error. How can I replace "depth.MapFromSkeletonPoint" for get a DepthImagePoint? I use SDK v1.6. Help me please

    • User profile image
      Rich Scheipe

      Getting 500 error when trying to download the samples at http://files.ch9.ms/coding4fun/KinectforWindowsSDKV1.zip
      I think these samples are needed to go through the Quickstart lessons.
      Thanks

    • User profile image
      nalikadamay​anthi

      Hi Dan,

      Do you have any examples of working Kinect sample codes witten in C++? I am doing a project related to working with multiple Kinects, so any tutorials or samples in C++ except SDK samples?

      Thanks,

      Nalika.

    • User profile image
      Lorenzo

      hello i run the example but i got a blank page. no errors occured.
      i got the xbox 360 kinect, maybe this is the problem?

    • User profile image
      Sifu104

      Thanks for an awesome introduction to the kinect. I am working with the Kinect in my bachelor thesis and I need it to be as accurate as possible. Can you calibrate it for more precious measurements? if yes how? 

    • User profile image
      kpcamota

      Hello everyone!

      aside from changing the color of the player, is there a possibility that I can get the distance and location or angle of the player using the player formula you had shown here? 

      I am new to kinect development. Thanks

    • User profile image
      shohin

      I tried to display the depth of a particular pixel (0,0) for example, i cudnt do it. Then i deleted the entire code and ran the program, the output did not change. How is this happening ? please tell me how to display the depth as a value

    Comments closed

    Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.