Here, Wolf Kienzle, Senior Research Developer, Interactive Visual Media group, Microsoft Research Redmond, explains and demos this excellent new tool for C++ developers building image, video or augmented reality apps. In effect, you can step into pixels. Excellent!

Blob Tracking: Kinect + OpenCV + WPF

With the latest SDK release, the Kinect platform has become pretty robust – skeleton tracking, gesture support, and access to the raw infrared stream all come “out-of-the-box”. Which is great, because if you’ve ever used any of the opensource equivalents, things can get pretty hairy.

Hands and skeletons are cool and all, but unfortunately there isn’t an *easy* way to do blob or generic object detection.. Right now, we’re working on a fun project and ultra fast blob detection is one of the major tech hurdles we’re working through.

After googlin’ around for blob-tracking with Kinect I came up empty handed. So I decided to look into OpenCV for doing blob detection. Many curse words later, I finally got things working the way I wanted. Along the way I noticed a handful of forums with people looking to do similar object tracking with Kinect, so I figured I would share how I cobbled things together.

Demo Overview & Source Code

The demo below is showing a couple of key things:

• Consuming the raw Depth image from the Kinect sdk
• Slicing the Depth image to a particular range
• Using OpenCV to perform object/blob tracking
• Rendering OpenCV output with WPF

...

Ok, so what?

I know what you’re thinking…so you have some blobs, big deal. And you might be wondering what else you can do with blobs. Well, let me tell you. By using generic blob detection you can not only detect when objects are present, you can also track their size, shape and rotation. For example, its possible to track an open hand transitioning to a closed fist for a grab gesture, or a user holding a wand or prop. Or in our case, we want to very efficiently capture the position of small, fast moving objects that penetrate an “invisible” IR plane. There are a ton of uses for blob detection including surveillance, card counting, and even real-time projection mapping.

What’s Next

While I’m pretty happy with the performance of this solution, I’m curious if it would be better/faster if I rewrote everything with the C++ flavor of the Kinect sdk and coupled it with DirectWrite or XAML to render the UI. Anybody out there do anything like this yet? If so, shoot me a tweet or email.

We’re currently rolling this stuff into a fun side project that I’ll post about shortly…stay tuned.

Project Information URL: http://blogs.claritycon.com/blog/2012/11/blob-tracking-kinect-opencv-wpf/

Project Download URL: https://github.com/eklimcz/OpenCV-WPF-Kinect

Project Source URL: https://github.com/eklimcz/OpenCV-WPF-Kinect

Contact Information:

• Blog: http://blogs.claritycon.com/blog/author/eklimczak/
• Twitter: @eklimcz

Using OpenCV 2 with Kinect SDK 1.5

OpenCV always makes a great combination with the Kinect SDK. I’m currently working on a project where I need to combine the two. Optical Flow and Face Recognition are just two out of many powerful features that complement the Kinect SDK.

I though this bit of code might be useful for anybody who wants to use these two frameworks together.

I’m just posting the whole class (C++) here (with links to the SDKs). Please feel free to ask any questions in the comments.

Project Information URL: http://www.rozengain.com/blog/2012/08/01/using-opencv-2-with-kinect-sdk-1-5/

Project Source URL: http://www.rozengain.com/blog/2012/08/01/using-opencv-2-with-kinect-sdk-1-5/

Contact Information:

• Blog: http://www.rozengain.com/blog/
• Twitter: @masdennis