Entries:
Comments:
Discussions:

Loading user information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading user information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Connecting your Netduino to your Kinect

Today's Hardware Friday post is a project that's a couple months old, but combines a couple of my favorite hardware things, the Kinect and Netduino.

Imagine adding NUI support to your next Netduino app...

KinectControlledNetduino

This code is an example of how the Kinect's gesture recognition features could be used to control a servo via a Netduino

Notes

David Catuhe has dome some hard work to add gesturing support on top of the Microsoft Kinect SDK. The links below describe how the gesturing library works and from where it can be downloaded.

This YouTube video demonstrates what the code in this repository will produce when fully used. * Demonstration YouTube video

Setup/Install Notes

    • Get the Netduino Plus connected to your machine via USB
    • Plug the Plus into the network via an Ethernet cable
    • Configure the Plus using MFDeploy to get the networking connection working
    • Hook a servo up to the Netduino Plus on one of the digital pins that supports PWM - 5,6,9, or 10 according to The Netduino Spec Page
    • Deploy the code to the Netduino Plus, or debug on it directly. Note, the default port is 8080, so the URL will be http://[PLUS.IP.Address.Here]:8080/servo/180 where 180 is the angle, between 0 and 180, to which you wish to have the servo turn.
      • Modify the Kinect GUI code so that it points to your Netduino Plus URL. Specifically, there is a line of code in the WebServoClient.cs file that reads string urlBase = "http://YOUR-NETDUINO-IP-HERE:8080/servo/{0}"; that you'll need to change.
    • Hook up your Kinect
    • Try to browse to your Netduino Plus URL until you see your servo respond.
    • Run the GUI code. You might want to tinker with the example code provided on the Kinect Toolkit CodePlex page to get a handle on the sensitivity, or disable the code in the GUI that calls the Netduino URL until you're comfortable gesturing and seeing the Kinect pick up your behavior on the screen.
    • If everything is communicating properly, you should see your Netduino Servo respond accordingly.

 

image

image

image

image

image

Note: If your MicroHTTP/MicroIO references are broken, you can find the DLL's in the WebServo\Libs folder.

Here's a snip of code;

    void OnGestureDetected(string gesture)
    {
        Debug.WriteLine(gesture);

        swipeLeft.Background = new SolidColorBrush(Colors.White);
        swipeRight.Background = new SolidColorBrush(Colors.White);

        if (gesture.Equals("SwipeToLeft", StringComparison.CurrentCultureIgnoreCase))
        {
            swipeLeft.Background = new SolidColorBrush(Colors.Red);
            this.client.TurnLeft();
        }

        if (gesture.Equals("SwipeToRight", StringComparison.CurrentCultureIgnoreCase))
        {
            swipeRight.Background = new SolidColorBrush(Colors.Red);
            this.client.TurnRight();
        }
    }

    void OnVideoFrameReady(object sender, ImageFrameReadyEventArgs e)
    {
        kinectDisplay.Source = streamManager.Update(e);
    }

    void OnSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
    {
        ProcessFrame(e.SkeletonFrame);
    }

    void ProcessFrame(ReplaySkeletonFrame frame)
    {
        Dictionary<int, string> stabilities = new Dictionary<int, string>();
        foreach (var skeleton in frame.Skeletons)
        {
            if (skeleton.TrackingState != SkeletonTrackingState.Tracked)
                continue;

            barycenterHelper.Add(skeleton.Position.ToVector3(), skeleton.TrackingID);

            stabilities.Add(skeleton.TrackingID, barycenterHelper.IsStable(skeleton.TrackingID) ? "Stable" : "Unstable");
            if (!barycenterHelper.IsStable(skeleton.TrackingID))
                continue;

            swipeGestureRecognizer.Add(skeleton.Joints
                .First(j => j.ID.Equals(JointID.HandRight)).Position, 
                    kinectRuntime.SkeletonEngine);

            postureRecognizer.TrackPostures(skeleton);
        }

        skeletonDisplayManager.Draw(frame);
    }
}

If you've been thinking of how to connect to your Kinect in your next Netduino project, this project shows you the way (at least one of them)

 

Here’s a few more links you might find interesting:

Follow the discussion

  • Oops, something didn't work.

    Getting subscription
    Subscribe to this conversation
    Unsubscribing
    Subscribing
  • Dan ThyerDan Thyer

    Really nice work! I used to work with Brady Gaster who is totally awesome!!

    You should also check out my Netduino / Kinect controlled squirt gun:
    http://www.youtube.com/watch?v=FWINsKcP8oQ

  • Hey that is actually pretty neat.  I like the green Forecolor command line on the other screen!

Remove this comment

Remove this thread

Close

Comments closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.