Real-Time Motion Capture using Kinect, XNA and the DigitalRune Engine
- Posted: Apr 19, 2012 at 6:00AM
- 13,056 views
Real-time motion capture is one of the more common requests I see for the Kinect so when I saw this...
Do note that this requires a third party project, the DigitalRune Engine, but you can get a free 30 trial for it AND if you're using this non-commercially you can get a free license too.
Our last post about real-time motion capture using Kinect seems to be pretty popular. And for the small amount of time (only a few hours) that we put into this example, the results are satisfying – but we can do better! We have updated the project and included a brand new sample. The new sample uses skeleton mapping to animate 3D models using Kinect. The code is much simpler, and exchanging the 3D model is a lot easier. We have also updated the example application to use Kinect SDK v1.0 instead of Kinect SDK beta2. You can download the sample project (including the source code) at the end of this post.
The new sample project demonstrates 2 different methods to animate 3D models using Kinect in real-time:
The Skeleton Mapping Approach
The first sample uses Skeleton Mapping to animate the XNA Dude model and a Space Marine model. There are three different skeletons involved: The Kinect player skeleton, the Dude skeleton and the Space Marine skeleton. All three skeletons are different, i.e. have different bone names and a different number of bones. DigitalRune Animation contains a SkeletonMapper which can be used to transfer an animation from one skeleton to another skeleton with a different structure. (Take a look at the documentation to learn more about Skeleton Mapping.)
Setting up the bone mapping for a model is not very difficult. This is the setup for the Dude model:
The source code, which can be downloaded below, contains additional comments and explanations.
The setup for the Space Marine model looks similar – only the bone names are different.
In each frame the SkeletonMapper is called to transfer the current pose of the Kinect player skeleton to the skeleton of the 3D model:
Kinect sensor input is not perfect and causes jittering. Jittering can be reduced by applying a low-pass filter to the bone orientations.
The Marionette Ragdoll Approach
The second example uses the approach that we have shown in the previous post: A physics ragdoll is created for the Dude, which is used to animate the model. Certain joints of the Kinect player skeleton are used as target positions (e.g. hands, elbows, head, knees, etc.). The ragdoll is attached to the target positions using weak ball joint constraints. – This is like a marionette: The ragdoll is the puppet and the ball joints are strings that pull on the marionette.
I was able to download the 30 trial, the sample code from the article (click through for it) and was playing with this in just a few minutes. Simple and painfully easy...
That's me (kind of)