Unity and the Kinect SDK
- Posted: Feb 27, 2012 at 6:00 AM
- 30,086 Views
- 33 Comments
Loading User Information from Channel 9
Something went wrong getting user information from Channel 9
Loading User Information from MSDN
Something went wrong getting user information from MSDN
Loading Visual Studio Achievements
Something went wrong getting the Visual Studio Achievements
Unity is is officially huge in the gaming industry, so when I saw this I thought that there had to be some of your that might find it interesting or useful...
Install the Unity Package
The pre-built Unity package includes all of the scripts required to start running your world.
To install, simply import the package, and you all of the required assets will appear in your project.
The following items are included within the package:
- KinectExample - This scene shows you how a skeleton is generated / tracked by placing spheres at each of the bones tracked by the Kinect, and how to use kinect to control your model. Use this to get a feel for what the Kinect is capable of. It also shows you how to prepare your GameObjects
- Kinect_Prefab- This prefab has an empty node which contains all of the necessary scripts for your world to start using the Kinect - this does NOT include the controller, which you will need to use to actually control your models. See setup
- KinectPointMan - This prefab shows how KinectPointController works.
- KinectModelControllerV2 - This is the script that you will attach to your model that you want to manipulate - you will drag each of the bones to be controlled into the appropriate slot, and determine which player controls which model.
- KinectPointController - This script will place GameObjects that you define onto points that are tracked by the Kinect, generating a skeleton. The starting scene comes with an example of how this looks / should be done.
- DisplayDepth - This script will get the depth image. Attach it to a game object with renderer. NOTE: In unity, you need to restart unity everytime after running your world, otherwise this script will not work (because of SDK issue).
- DisplayColor - This script will get the RGB image. Attach it to a game object with renderer. NOTE: In unity, you need to restart unity everytime after running your world, otherwise this script will not work (because of SDK issue).
- KinectRecorder - This script will record your movement and output playback files for the emulator.
- KinectEmulator - This script will act as a virtual Kinect. It works with playback files. For now it only simulate the skeleton data.
- KinectSensor - This script gets data from the physic Kinect.
- DeviceOrEmulator - This script sets whether to use physic Kinect or the emulator.
- SkeletonWrapper - This script grabs skeleton data.
- DepthWrapper - This script grabs depth image data.
- KinectInterop - This script grabs data from Microsoft Kinect SDK.
- Recordings/playbackDefault - This is the default playback file for emulator. Do NOT remove this file.
Models and Materials
- This is an example model that you can use for testing / learning how to use the wrapper. You can drag the KinectModelControllerV2 script onto this model and attach the bones and immediately begin testing.
Using the Scripts
After importing the asset package, you have the KinectExample to test and get familiar with the KinectPointController and KinectModelControllerV2. You can use this as your base scene, or you can drag the appropriate GameObjects into a new one.
Setting up the Kinect_Prefab
As previously mentioned, the Kinect_Prefab GameObject is required to have Unity talk with / use the Kinect. This empty object needs to exist somewhere in your scene. However, if you are switching scenes, it will persist because of the DontDestroyOnLoad call in the KinectWrapper script, so you don't need to instantiate a new Kinect_Prefab in each scene.
To get the best use out of the Kinect, you may need to tweak some settings within the Kinect_Prefab object itself - here is a breakdown of each option.
Device Or Emulator
- Use Emulator - Check this to use emulator.
NOTE: do NOT enable this script manually, it is controlled by DeviceOrEmulator.
- Sensor Height - How high (in meters) off the ground is the sensor.
- Kinect Center - This tells the Kinect where it should be looking for it's 0,0,0 point (relative to the ground directly under the sensor). The default works pretty well.
- Look At - Tells the Kinect how to orient the camera using the motor control.
NOTE: The following values allow you to smooth the skeleton data. Usually the default values work fine. Do NOT change them unless you find magic numbers for you project.
- Smoothing - Default value 0.5.
- Correction - Default value 0.5.
- Prediction - Default value 0.5.
- Jitter Radius - Default value 0.05.
- Max Deviation Radius - Default value 0.04.
Project Information URL: http://wiki.etc.cmu.edu/unity3d/index.php/Microsoft_Kinect_-_Microsoft_SDK
This project looks to be using the beta version of the SDK and not v1 (as of the time of this post). With the release of the Kinect for Windows SDK v1, I've meant to only highlight projects that are v1 compatible, but missed a few phrase on the page, "Also, since we are not using the new version of Kinect SDK, do NOT download it from Microsoft website...."
I would NOT recommend installing the Kinect for Windows SDK from any other source than Microsoft and so suggest, if you've already upgraded to v1 for the SDK, that you hold off going to far with this project. If you are interested in the project, because it does look pretty darn cool, then you encourage the authors to upgrade to v1 as soon as they can.