Kinect for Windows SDK 1.5 - Face Tracking, Seated Skeletal Tracking, Kinect Studio, & More

Download this episode

Download Video

Description

Rob Relyea, a Principal Program Manager on the Kinect for Windows team joins us again on Channel 9 to discuss all of the new features with the 1.5 release of the Kinect for Windows SDK.

[00:33] Overview of the Kinect for Windows SDK 1.5 Release

[03:00] Improvements to the Kinect Explorer Sample

[03:38] Using the Kinect Explorer sample with Seated Skeletal Tracking

[04:37] Kinect Avatar/XNA sample that demonstrates the new support for skeletal joint orientation

[06:47] Face Tracking sample that locates and orients a face, along with 70+ vertices to track eyes, eyebrows, mouth, etc

[08:49] Kinect Studio developer tool that shows how to record and playback Kinect data in your application

[11:26] Green screen sample shows background separation 

[12:03] Rob discusses the improved synchronization of depth and color data and improved performance in mapping color to depth pixels 

[13:34] Direct 3D Demo that shows a 3D Point cloud visualization of Kinect data

[14:50] Speech Basics demo for navigating a turtle by voice

[15:40] Kinect language packs now support English (US, Australia, Canada, Great Britain, Ireland, New Zealand, French (Canada and France), Spanish (Spain, Mexico), Italian, and Japanese

Embed

Format

Available formats for this video:

Actual format may change based on video formats available and browser capability.

    The Discussion

    • Rob Relyea

      Happy to be back on Channel 9 with details of our SDK and Developer Toolkit v1.5. They are downloadable now.

      We hope you enjoy!

      -Rob Relyea, @rrelyea
      Program Manager, Kinect for Windows team

       

    • k4gdw

      I'd really like to see Kinect for Windows support in Windows 8.  It would make the Metro interface actually usable on the desktop where a touch screen really isn't practical.  Imagine a "Minority Report" like experience for all of the swiping gestures.

    • Richard.Hein

      Oh this is awesome.  Big Smile  I need to get the hardware. 

    • jaaouaneyas​sine

      @k4gdw: look to our post about using kinect with windows 8 metro style app:

      http://channel9.msdn.com/coding4fun/kinect/Kinect-your-Metro

    • El Bruno

      Great release !!! Right now I'm upgrading some applications .. let's see some improvements Big Smile Big Smile Big Smile

    • atif980

      Wow this really is amazing.  Huge Giggle  I have to aquire the actual hardware.

    • Mark

      Hello

      It's a great notice. I have one question, which is better, program applications in C + + or c #? I want to start working with the SDK Kinect and do not know which is better.
      El Bruno, you who have done so many examples online?
      Thank you very much.

      Greetings.

    • aL3891

      This is really awesome, face tracking And seated mode!

      Do you have plans for doing a similar ST - depth combo for the hands as well?

    • wowps

      Hello from Singapore. The 'Kinect for Windows' has been released in Singapore, and I am going to buy it. I have one question though - can I use the Kinect I get with 'Kinect for Windows' with XBox to play games?

      I need to have fun while I program Smiley

      Thanks for reply.

    • mark

      Hey.

      I want to create applications for detection of the skeleton of the face. But I have a problem, do not understand how the examples are made sdk brings kinect. Do as the generated DLL? That is, how do you program? Thank you very much everyone for the help.

      Greetings.

    • Rob Relyea

      @Mark-
      I'd recommend you consider which UI Framework will meet your application needs, and pursue languages that work with that UI Framework.  Our Toolkit contains samples using WPF, XNA, Direct3D, and Direct2D. Once you choose the framework that will meet your needs, the language choice is clear.

      @aL3891-
      Nothing to announce about fingers. Common request.

      @wowps-
      Kinect for Windows sensors may not be used with an Xbox 360.
      Kinect for Xbox sensors may not be used in licensed deployments on Windows (they may be used by Devs...but we strongly recommend devs migrate towards Kinect for Windows sensors).

      @mark-
      You may want to try this bing search for "C# programming for beginners"

      Thanks all!

      -Rob

    • Hemanth

      I am working on gesture recognition with depth data. I have a set of RGB and depth videos collect from a kinect. Would it be possible to use the SDK to extract skeleton information for the gestures in these prerecorded videos. I looked up the examples on the SDK but it does not seem to support prerecorded videos. Please let me know if there is a way around this problem.
      Thank you.
      Hemanth

    • Vicent

      Hello,
      Face Tracking is great!!
      But I have performance problems when I use it with XNA, the fps rate fall down so much... Have you had troubles with that issue??

    • Rob Relyea

      @Hemanth: Our runtime can either use live data from the sensor, or via Kinect Studio, pre-recorded color and depth feeds. Kinect Studio is a new Tool that ships in the Kinect for Windows Toolkit 1.5.0. It allows you to record and playback these feeds.

      When you play them back, they are in your application just like a live feed, so you can see the Skeletal Tracking data, etc...

      Thanks, Rob

    • Rob Relyea

      @Vicent: I've not heard of this issue yet. Can you please engage with our team in our developer forums - http://bit.ly/KinectSDKForums. We've just created a new subforum covering face tracking.

      Also...take a look at the XnaBasics sample (under C# samples) to see that you make sure to set:

                  this.IsFixedTimeStep = false;

      Thanks, Rob

    • Hemanth

      @Rob: Thank you for your reply. Appreciate it very much. I just had a couple of questions more. Are there any examples to read a .xed file (Kinect studio stores depth and color as .xed files) and extract skeleton information. How can I write my own .xed file with a pre-recorded depth stream and color stream.

      Thank you very much
      Hemanth

    • Rob Relyea

      @Hemanth:We haven't yet exposed APIs for Kinect Studio .XED files or documented the format. So, for now, sorry, you can't import that data into .XED file format.

    • Hemanth

      @Rob: Thank you once again. Appreciate your response. I understand that I may not be able to convert a prerecorded depth and color stream into .XED format. I am working on a gesture-recognition project. It has prerecorded depth and color videos but there is no skeleton stream available. I would like to know if there is any other way to extract skeleton information from prerecorded depth and color videos. I presume that gesture recognition will be easier with skeleton information.

      Thank you very much.
      -Hemanth

    • KNIect

      @Rob: is there any draft documentation on the XED format ? I'd be interested in converting existing recordings from a massive database to this new format, as Kinect Studio is really nice.

    • Rob Relyea

      @Hermanth: Sorry. Right now, you can only get skeletal information two ways: 1) live data from the Kinect Sensor; 2) playback of recorded color/depth data via Kinect Studio. Thx, Rob

    • Rob Relyea

      @KNIect: Glad you like Kinect Studio. Sorry, there is no draft documentation of the .XED file format.

      Thx, Rob

    • HAhmed

      Kinect studio and Developer Toolkit is great!

    • amir

      Hi Rob,
      Is there any way to convert the .XED file into color and depth frame sequence separately. For example, maybe save in .jpeg sequence.

    Comments closed

    Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.