A Very Kinect Christmas

While Christmas is behind us and the decorations put away, it's about that time to start going through our pictures. And there's few cuter pictures than this (plus it's a cool Kinect project too

[#KINECT] Merry Christmas with Kinect and Visual Studio 2010 (gift of the Valentino)

In these days of festivities, when you join the side 2 dwarves wanting to play, you got the kinect connected to the PC and the SDK that you strange, at the very least you get is something like the following:

Disclaimer: as they can be seen in the screenshot, in addition to the view of the WebCam I am showing the skeleton, but just as he gave a ray of sunshine to my dwarf in the arm, therefore not recognized him!

As well, after several post about Kinect, today the challenge was as follows

1. identify the specific joint of the head of each skeleton
2. calculate the relative coordinates of that joint before an image of the webcam
3. paint a Santa Hat on webcam

The first point is easily solved once we iteramos by the skeletons recognized in mode Trackeable (lines 6 to 9), we can iterate between the Joints of the skeleton and verify that it is the head compared with [JointID.Head] (line 17).

Once identified the head Joint we come to the 2nd point which is a bit more complicated. Function GetDisplayPosition() (line 27) is in charge for the conversion of the position of the Joint to relative location in the image. The following capabilities of the SDK are used to this

- SkeletonEngine.SkeletonToDepthImage () using this function we can know the relative location in “depth” of a joint.

- NuiCamera.GetColorPixelCoordinatesFromDepthPixel () using this function, we can discover the position at coordinates x/Y from the depth we obtained earlier.

Project Information URL: http://elbruno.com/2011/12/28/kinect-merry-christmas-with-kinect-and-visual-studio-2010-gift-of-the-valentino/, http://kinectmerrychristmas.codeplex.com/

Project Download URL: http://kinectmerrychristmas.codeplex.com/releases/view/79524

Project Source URL: http://kinectmerrychristmas.codeplex.com/SourceControl/list/changesets

Runs with Beta2? Yes

void KinectSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
    canvasSkeleton.Children.Clear();
    this.HideImageHats();
    var skeletonId = 0;
    foreach (var skeleton in e.SkeletonFrame.Skeletons.Where
                  (skeleton =>
                            SkeletonTrackingState.Tracked == skeleton.TrackingState))
    {
        skeletonId++;
        this.DisplayImageHats(skeletonId);
        PaintBones(skeleton);
        PaintJoints(skeleton);

        foreach (Joint joint in skeleton.Joints)
        {
            if (joint.ID != JointID.Head)
            {
                continue;
            }


            float depthX;
            float depthY;
            kinect.SkeletonEngine.SkeletonToDepthImage(joint.Position, out depthX, out depthY);
            var depthPixelX = (int)Math.Max(0, Math.Min(depthX * CameraViewer.ActualWidth, CameraViewer.ActualWidth - 1));
            var depthPixelY = (int)Math.Max(0, Math.Min(depthY * CameraViewer.ActualHeight, CameraViewer.ActualHeight - 1));

            depthPixelY -= 20;

            var newPosition = this.GetDisplayPosition(joint);
            this.MoveImageHats(newPosition, skeletonId);

            Debug.WriteLine(string.Format("{0} ", DateTime.Now.TimeOfDay));
            Debug.WriteLine(string.Format("    X:{0} Y:{1}", joint.Position.X, joint.Position.Y));
            Debug.WriteLine(string.Format("    W:{0} Z:{1}", joint.Position.W, joint.Position.Z));
            Debug.WriteLine(string.Format("    SkeletonToDepthImage depthX:{0} depthY:{1}", depthX, depthY));
            Debug.WriteLine(string.Format("    depthPixelX:{0} depthPixelY:{1}", depthPixelX, depthPixelY));
            Debug.WriteLine(string.Format("    New position X:{0} Y:{1}", newPosition.X, newPosition.Y));
        }
    }
}

private Point GetDisplayPosition(Joint joint)
{
    float depthX, depthY;
    kinect.SkeletonEngine.SkeletonToDepthImage(joint.Position, out depthX, out depthY);
    depthX = Math.Max(0, Math.Min(depthX * 320, 320));  //convert to 320, 240 space
    depthY = Math.Max(0, Math.Min(depthY * 240, 240));  //convert to 320, 240 space
    int colorX, colorY;
    var iv = new ImageViewArea();
    // only ImageResolution.Resolution640x480 is supported at this point
    kinect.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(ImageResolution.Resolution640x480, iv, (int)depthX, (int)depthY, (short)0, out colorX, out colorY);
    //colorX = colorX / 2;
    //colorY = colorY / 2;

    // map back to skeleton.Width & skeleton.Height
    var newX = (int)(CameraViewer.ActualWidth * colorX / 640.0) - 30;
    var intY = (int)(CameraViewer.ActualHeight * colorY / 480) - 30;
    return new Point(newX, intY);
}

private void MoveImageHats(Point newPosition, int skeletonId)
{
    if (skeletonId == 1)
    {
        Canvas.SetLeft(this.contentControlHat1, newPosition.X);
        Canvas.SetTop(this.contentControlHat1, newPosition.Y);
    }
    if (skeletonId == 2)
    {
        Canvas.SetLeft(this.contentControlHat2, newPosition.X);
        Canvas.SetTop(this.contentControlHat2, newPosition.Y);
    }
}

private void DisplayImageHats(int skeletonId)
{
    if (skeletonId == 1)
    {
        this.imageHat1.Visibility = Visibility.Visible;
    }
    if (skeletonId == 2)
    {
        this.imageHat2.Visibility = Visibility.Visible;
    }
}

Contact Information:

• Blog: http://elbruno.com
• Twitter: @ElBruno