A quick view behind the curtain at the XBox 360's Kinect Tuner
Today's Gallery post is a little different than most. While I'm usually focused on the Kinect for Windows SDK, when I saw this, while its not directly related to the SDK, I thought it interesting and since it is related to the Kinect and the magic behind it.
I mean we all want to know how the XBox 360 and Kinect works its magic right? This post, though a from early this year, from the XBox Live Engineering Blog reveals some of the story behind the Kinect Tuner.
Introducing … the Kinect Tuner!
In this post, I will mostly discuss the internals and history of the Tracking Tuner, touch a bit on the Audio Tuner, and discuss some tips and tricks for getting the most out of both. The Depth Calibration part of the Tuner will be covered in a later post.
The Evolution of the Tracking Tuner
One might think that software development would follow a blueprint, similar to the construction of a bridge. An engineer lays out exactly what it will look like, and then the construction crew comes along and makes it happen.
But with the Kinect Tuner, as with many software projects, the development followed more of an evolutionary cycle. Here’s the path we took, and some interesting stops we made along the way.
The Primordial Sludge
When our team started working on the tracking part of the Kinect Tuner (then known as the “Natal Troubleshooter” or “Nui Troubleshooter”), we had no idea what it was going to be. We just knew there needed to be some system-wide way for people to figure out their tracking issues.
Working in the Guide is quite limiting because it gets only a small portion of the Xbox 360’s resources – the rest understandably go to the currently running game. Fortunately, with coding magic, we were quickly able to get a depth map feed displaying in real time in the Guide (albeit with very little smoothing or other graphical enhancements), which proved that what we wanted was in the realm of possibility.
Dinosaurs and Mammals
Once we had something running, we tried a few different approaches.
We spent quite a bit of time trying to give the Kinect Tuner the look and feel of the current Out Of Box Experience (OOBE) – the little play space setup that you do when you plug in Kinect for the first time. OOBE has an interesting approach – they put a virtual “camera” behind the user, so the screen shows the user’s avatar (from behind) with a mockup of the TV in the background.
While figuring out how to represent the play space, we were also working on how to automatically “test” whether the user was well positioned within the play space and could be seen clearly. Once we had settled on using the depth map, the team was able to focus much more on this part. We ended up using tech provided by multiple teams working on portions of the Kinect platform in order to test different aspects of the play space.
Some tests didn’t end up making the final cut because they were unreliable or unnecessary. For example, at one point we had a test that checked whether your body was being obscured. It turned out that this frequently failed due to the user crossing their arms, so it had to go.
Project Information URL: http://www.xbox.com/en-us/Live/EngineeringBlog/122910-kinect-tuner