Channel 9

Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

MSDN

Activity Profile

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements
Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Skeletal Tracking Fundamentals (Beta 2 SDK)

Download

How do I download the videos?

  • To download, right click the file type you would like and pick “Save target as…” or “Save link as…”

Why should I download videos from Channel9?

  • It's an easy way to save the videos you like locally.
  • You can save the videos in order to watch them offline.
  • If all you want is to hear the audio, you can download the MP3!

Which version should I choose?

  • If you want to view the video on your PC, Xbox or Media Center, download the High Quality WMV file (this is the highest quality version we have available).
  • If you'd like a lower bitrate version, to reduce the download time or cost, then choose the Medium Quality WMV file.
  • If you have a Zune, WP7, iPhone, iPad, or iPod device, choose the low or medium MP4 file.
  • If you just want to hear the audio of the video, choose the MP3 file.

Right click “Save as…”

Embed code for this video

Copy the code above to embed our video on your website/blog.

Close

Video format

Note: These selections will fall back to the next best format depending upon browser capability.

Close

Update: Kinect for Window SDK v1 Quickstart Series now Available (Feb 1st)

Please use the newly updated Kinect for Windows SDK Quickstart series. The content below will only work with the Beta 2 version of the Kinect for Windows SDK.

 

This video covers the basics of skeletal tracking using the Kinect sensor.  You may find it easier to follow along by downloading the Kinect for Windows SDK Quickstarts samples and slides that have been updated for Beta 2 (Nov, 2011).

  • [00:31] Skeleton Tracking API
  • [01:24] Understanding Skeleton Quality and Joint data
  • [03:27] Setup skeleton tracking
  • [03:44] Adding a basic hand tracked cursor
  • [09:12] Using TransformSmoothing to remove “skeletal jitter”

 

Updates for Kinect for Windows SDK Beta 2 (Nov, 2011)

The video has not been updated for Beta 2, but the following changes have been made:

  • Previously, the SkeletonFrameReady event would only fire when there was a tracked skeleton. The SkeletonFrameReady has been updated to fire continuously even when there are no skeletons being tracked. To get tracked skeletons, you can filter the list of skeletons to see if the TrackingState property is set to SkeletonTrackingState.Tracked.
  • Beta 2 includes a sample (Microsoft.Samples.Kinect.WpfViewers) set of user controls for visualizing data from the Color/RGB and Depth cameras that you can reuse in your application. The samples can be found at: "\Program Files\Microsoft SDKs\Kinect\v1.0 Beta2\Samples\KinectSDKSamples.zip"

 

Setup

The steps below assume you have setup your development environment as explained in the Setting Up Your Development Environment video.

Task: Setup skeleton tracking

Create the Window_Loaded event

Go to the properties window (F4), select the MainWindow, select the Events tab, and double click on the Loaded event to create the Window_Loaded event

image

 

Initializing the runtime

Create a new variable outside of the Window_Loaded event to reference the Kinect runtime.

C#

Runtime nui = new Runtime();

Visual Basic

Dim nui As New Runtime()

In the Window_Loaded event, initialize the runtime with the options you want to use. For this example, set RuntimeOptions.UseSkeletalTracking to receive skeletal data and register for the SkeletonFrameReady event.

C#

nui.Initialize(RuntimeOptions.UseSkeletalTracking);

nui.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);

void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
}

Visual Basic

nui.Initialize(RuntimeOptions.UseSkeletalTracking)

AddHandler nui.SkeletonFrameReady, AddressOf nui_SkeletonFrameReady

Private Sub nui_SkeletonFrameReady(ByVal sender As Object, ByVal e As SkeletonFrameReadyEventArgs)

End Sub

Running the application

Add a breakpoint inside the SkeletonFrameReady event and run the application.

Note – You will need to stand far enough away that the Kinect can see all or most of your skeleton for the SkeletonFrameReady event to fire.

image

 

When the breakpoint fires, you can inspect the SkeletonFrameReadyEventArgs to see that the SkeletonFrame returns a collection of six skeletons as shown below.

 

image

 

 

Task: Add a basic hand tracked cursor

In this example, we're going to use the position of the tracked skeleton's, head, left hand, and right hand to move ellipse controls.

Designing your UI

Starting from the project above, switch to MainWindow.xaml and make sure you can see the XAML code. We will add three three ellipses of varying color onto a Canvas control named MainCanvas as shown in the XAML below.

XAML

<Canvas Name="MainCanvas">
    <Ellipse Canvas.Left="0" Canvas.Top="0" Height="50" Name="headEllipse" Stroke="Black" Width="50" Fill="Orange" />
    <Ellipse Canvas.Left="50" Canvas.Top="0" Height="50" Name="rightEllipse" Stroke="Black" Width="50" Fill="SlateGray" />
    <Ellipse Canvas.Left="100" Canvas.Top="0" Fill="SpringGreen" Height="50" Name="leftEllipse" Stroke="Black" Width="50" />
</Canvas>

Get the Tracked Skeleton

In the SkeletonFrameReady event, we'll use a LINQ query to get the first tracked skeleton.

C#

void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
    SkeletonFrame allSkeletons = e.SkeletonFrame;

    //get the first tracked skeleton
    SkeletonData skeleton = (from s in allSkeletons.Skeletons
                             where s.TrackingState == SkeletonTrackingState.Tracked
                             select s).FirstOrDefault();
}

Visual Basic

Private Sub nui_SkeletonFrameReady(ByVal sender As Object, ByVal e As SkeletonFrameReadyEventArgs)
    Dim allSkeletons As SkeletonFrame = e.SkeletonFrame

    'get the first tracked skeleton
    Dim skeleton As SkeletonData = ( _
        From s In allSkeletons.Skeletons _
        Where s.TrackingState = SkeletonTrackingState.Tracked _
        Select s).FirstOrDefault()
End Sub

Getting a Joint position

A Joint position returns X,Y,Z values as explained below

  • X = Horizontal position measured as the distance, in meters from the Kinect along the X Axis
  • Y = Vertical position measured as the distance, in meters from the Kinect along the Y Axis
  • Z = Distance from Kinect measured in meters

Scaling a Joint value

Given that the X and Y positions are distance measurements, we can use the Coding4Fun Kinect Toolkit ScaleTo method to scale the value to a maximum X and Y position as shown below.

C#

Joint HandRight =  skeleton.Joints[JointID.HandRight].ScaleTo(640, 480);

Visual Basic

Dim HandRight = skeleton.Joints(JointID.HandRight).ScaleTo(640, 480)

You can also constrain the maximum values for the skeletal joints to a smaller range.  For example, if you were designing an application where the user can reach around the screen to touch things, you may not want them to have to walk two feet left or right to reach the edges.  By using the second version of the ScaleTo method, you can specify the range of the specified joint to something smaller as shown:

C#

Joint HandRight =  skeleton.Joints[JointID.HandRight].ScaleTo(640, 480, .5f, .5f);

Visual Basic

Dim HandRight = skeleton.Joints(JointID.HandRight).ScaleTo(640, 480, .5f, .5f)

This code will make it so that your right hand only has to travel one meter (joint range of -0.5 to +0.5) to traverse the pixels at 0 to 640. If you set this value to –1 to +1, for example, then the right hand would have to travel two meters total, one meter to the left, one meter to the right to traverse the 640 pixels.

Also, as you can see above, you can get a particular Joint, like the HandRight Joint, by using the skeleton indexer for the Joints collection.

Setting an Ellipse Position

To move the ellipses in our MainWindow  to the location of a Joint, we will use the method below that sets the Canvas.Left and Canvas.Top position to the X (Left) and Y (Top) value from the Joint parameter.

C#

private void SetEllipsePosition(FrameworkElement ellipse, Joint joint)
{
    var scaledJoint = joint.ScaleTo(640, 480, .5f, .5f);

    Canvas.SetLeft(ellipse, scaledJoint.Position.X);
    Canvas.SetTop(ellipse, scaledJoint.Position.Y);
}

Visual Basic

Private Sub SetEllipsePosition(ByVal ellipse As FrameworkElement, ByVal joint As Joint)
    Dim scaledJoint = joint.ScaleTo(640, 480,.5f,.5f)

    Canvas.SetLeft(ellipse, scaledJoint.Position.X)
    Canvas.SetTop(ellipse, scaledJoint.Position.Y)
End Sub

 

Putting it all together

Finally, in the nui_SkeletonFrameReady event, call the SetEllipsePosition methods for the three joints as shown below:

C#

SetEllipsePosition(headEllipse, skeleton.Joints[JointID.Head]);
SetEllipsePosition(leftEllipse, skeleton.Joints[JointID.HandLeft]);
SetEllipsePosition(rightEllipse, skeleton.Joints[JointID.HandRight]);

Visual Basic

SetEllipsePosition(headEllipse, skeleton.Joints(JointID.Head))
SetEllipsePosition(leftEllipse, skeleton.Joints(JointID.HandLeft))
SetEllipsePosition(rightEllipse, skeleton.Joints(JointID.HandRight))

image

 

 

Task: Using TransformSmoothing to for less skeletal jitter

Using the same application as above, you may notice jitteriness in the hand positions as small changes between updates to the SkeletalFrameReady event change the location of the ellipses.

Using TransformSmoothParameters

To see the difference between using and not using TransformSmoothing, toggle the true/false TransformSmooth property.  There are two ways to use TransformSmoothing, you can set it to true and it will use a default set of parameters, or you can customize and experiment with each of the parameters to find the parameters that work best for your application. To do that, you’ll need create the TransformSmoothParameters struct yourself and define the parameters.

You must set the TransformSmoothParameters after calling the nui.Initialize method.

Note: Since every application is different, you will need to experiment with the parameters to understand what's right for your application.

C#

private void SetupKinect()
{
     if (Runtime.Kinects.Count == 0)
     {
         this.Title = "No Kinect connected"; 
     }
     else
     {
         //use first Kinect
         nui = Runtime.Kinects[0];
         //Initialize to do skeletal tracking
         nui.Initialize(RuntimeOptions.UseSkeletalTracking);
         //add event to receive skeleton data
         nui.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);
         //to experiment, toggle TransformSmooth between true & false
         // parameters used to smooth the skeleton data
         nui.SkeletonEngine.TransformSmooth = true;
         TransformSmoothParameters parameters = new TransformSmoothParameters();
         parameters.Smoothing = 0.7f;
         parameters.Correction = 0.3f;
         parameters.Prediction = 0.4f;
         parameters.JitterRadius = 1.0f;
         parameters.MaxDeviationRadius = 0.5f;
         nui.SkeletonEngine.SmoothParameters = parameters;
     }
}

Visual Basic

        Private Sub SetupKinect()
            If Runtime.Kinects.Count = 0 Then
                Me.Title = "No Kinect connected"
            Else
                'use first Kinect
                nui = Runtime.Kinects(0)
                'Initialize to do skeletal tracking
                nui.Initialize(RuntimeOptions.UseSkeletalTracking)
                'add event to receive skeleton data
                AddHandler nui.SkeletonFrameReady, AddressOf nui_SkeletonFrameReady
                'to experiment, toggle TransformSmooth between true & false
                ' parameters used to smooth the skeleton data
                nui.SkeletonEngine.TransformSmooth = True
                Dim parameters As New TransformSmoothParameters()
                parameters.Smoothing = 0.7f
                parameters.Correction = 0.3f
                parameters.Prediction = 0.4f
                parameters.JitterRadius = 1.0f
                parameters.MaxDeviationRadius = 0.5f
                nui.SkeletonEngine.SmoothParameters = parameters
            End If
        End Sub

Comment on the Post

Already have a Channel 9 account? Please sign in

More posts in this series

See More

Creative Commons License

© 2012 Microsoft. Except where designated as licensed by
Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License,
Microsoft reserves all rights associated with the materials on this site.