Kinecting with Robots - Kinect Services for the Microsoft Robotic Development Studio

Description

Sight

That's what my robot needs, sight, human like, depth perception, sight. And of course, as a developer, I need the capability to program my robot to use that sight.

Hum... Eyes... Sight... Depth sensing... sounds like this cool device I've heard so much about. Now if only I could use with the free cool Microsoft Robotic Development Studio 2008 R3 ...

Kinect Services for RDS 2008 R3

Kinect Services for RDS provides sample services that use the Kinect for Windows SDK to allow access to the Depth and RGB data from a Kinect sensor. In addition to a service for a real Kinect, there is also a service for a simulated Kinect that works with the RDS simulator. A sample application is included that shows how to use a Kinect on a simulated robot to wander around and avoid obstacles. Note that both Visual Studio 2008 and 2010 are required to compile the Kinect services. This release is for RDS 2008 R3 only.

These are exciting times for Robotics. It is clear that the convergence of technologies and secular trends will make Robotics relevant to our day to day lives. The availability of Kinect and the work that the Robotics community has been doing the last few months is an example of the energy and the potential that exists in this space. You may have already seen in the news a couple of weeks ago that Microsoft released our Kinect for Windows SDK Beta (Kinect SDK) —creating opportunities in exciting ways by making advanced technologies, such as skeleton tracking, available to everyone.

Today, I’m pleased to announce that we are releasing Kinect Services for RDS, available as a free download from our website. Kinect Services for RDS is built on top of the Kinect for Windows SDK. We have wrapped in the core Kinect SDK capabilities so you can use them with the CCR/DSS programming model. However, this is only the beginning. Keep your eyes peeled for updates in the fall. We know that there are many practical issues for applying the Kinect technologies to robotics capabilities that we will be addressing in upcoming RDS releases.

Note: Make sure you read the "KinectServicesForRDS2008R3.docx" document that comes in the download. This will walk you through all the steps you need to take to make this work, how to run the samples, the doc's for the API/implementation, etc, in short, RTFD (Read The Fine Doc)

Project Information URL: http://blogs.msdn.com/b/msroboticsstudio/archive/2011/07/13/announcing-kinect-services-for-rds-2008-r3.aspx

Project Download URL: http://research.microsoft.com/en-us/downloads/f8cda115-e9ec-44d1-abcd-3dfdd09d2e77/

image

image

image

image

imageimage

image

private void InterpretDepthImage(Microsoft.Robotics.Services.DepthCamSensor.DepthCamSensorState depthState, out double forwardSpeed, out double rotationSpeed)
{
    // Use the depth data to find a clear path. Adjust forward speed according to the nearest obstacle ahead. Adjust rotation speed to find a clear area.

    // create this bitmap if necessary
    if (this.colorCodeBitmap == null)
    {
        this.colorCodeBitmap = new Bitmap(depthState.DepthImageSize.Width, depthState.DepthImageSize.Height);
    }

    if (this.sendImage)
    {
        SendDebugImage(depthState);
    }

    // holds intermediate results for the approximate clear area in each column of the depth data
    short[] clear = new short[depthState.DepthImageSize.Width];

    // calculate the furthest "clear" distance for each column in the depth image
    for (int x = 0; x < depthState.DepthImageSize.Width; x++)
    {
        short furthestFloor = 0;
        short nearestObstacle = (short)constants.KinectReservedSampleValues.MaxValidDepth;

        // start at the bottom and keep track of the furthest floor sample and the closest non-floor sample
        for (int y = depthState.DepthImageSize.Height - 1; y > 0; y--)
        {
            short depthSample = this.GetDepthSample(depthState.DepthImage, depthState.DepthImageSize.Width, x, y);

            // disregard samples that have no reading
            if (depthSample == (short)constants.KinectReservedSampleValues.PrefilterNoReading)
            {
                continue;
            }

            if (this.SampleIsFloor(depthSample, this.GetDepthSample(this.floorDepthData, depthState.DepthImageSize.Width, x, y)))
            {
                furthestFloor = Math.Max(furthestFloor, depthSample);
            }
            else
            {
                nearestObstacle = Math.Min(nearestObstacle, depthSample);
            }
        }

        // keep the closest distance
        clear[x] = Math.Min(furthestFloor, nearestObstacle);

        // If the entire column is invalid data, mark it as no clearance
        if (clear[x] >= (short)constants.KinectReservedSampleValues.MaxValidDepth)
        {
            clear[x] = 0;
        }
    }

    // save this in the state so it can be easily retrieved and viewed in the browser
    this.state.ClearDistances = clear;

    // find the closest distance for each segment of width kernelWidth
    var summary = new short[clear.Length - this.kernelWidth];
    for (int i = 0; i < summary.Length; i++)
    {
        summary[i] = (short)constants.KinectReservedSampleValues.MaxValidDepth;
        for (int j = i; j < (i + this.kernelWidth); j++)
        {
            summary[i] = Math.Min(summary[i], clear[j]);
        }
    }

    var centerIndex = summary.Length / 2;
    var direction = centerIndex;

    // check the straight-ahead distance. If there is an obstacle approaching, slow down
    // and look for a more open path to the right or left.
    
    // special case for clear path ahead
    if (summary[centerIndex] > ClearDistance)
    {
        forwardSpeed = 1.0;
        rotationSpeed = 0.0;
        return;
    }

    // some obstacle ahead, adjust speed and rotation
    forwardSpeed = (summary[centerIndex] - StopDistance) * RangeReciprocal;
    forwardSpeed = Math.Max(forwardSpeed, 0);
    for (int offset = 1; offset < centerIndex; offset++)
    {
        if (summary[centerIndex + offset] > ClearDistance)
        {
            // smooth sailing
            direction = offset;
            break;
        }

        if (summary[centerIndex - offset] > ClearDistance)
        {
            // smooth sailing
            direction = -offset;
            break;
        }
    }

    rotationSpeed = (double)direction / (double)centerIndex;
}

Contact Information:

Blog: Microsoft Robotics Blog

 

The Discussion

  • User profile image
    ackleybooth

    At the end of things, the wall mounting product is most likely the perfect selection for the greatest amount of users of this Xbox peripheral, even if you’re only guessing at probable space allocation based on an Xbox 360 Kinect trailer. However, you’ll probably wish to make sure you aren’t a perfect match with one of the other two products, since they are able to work competently, within their scopes. And either way, be ready for a lot of motion gaming fun!

Comments closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to send us feedback you can Contact Us.