"Analysis of the Kinect For Windows SDK 1.8 Background Removal API"


The second post in our mini-Background Removal theme week comes from someone I don't believe I've highlighted before, but I think that's going to change... Wink

Charles Palen takes a closer look at the official Background Removal, and having done the same thing himself, discusses how he thinks it does its magic...

Analysis of the Kinect For Windows SDK 1.8 Background Removal API


This summer for the Kinect Green Screen Photo kiosk I had at Maker Faire Detroit, I invested about a month of time figuring out how to get a good mask out of the Kinect in real time. The approach I used was to ignore skeleton data, and only use depth data, then run that raw data through EMGUCV(.Net OpenCV) and do blob detection, take the detected blobs, and run them through a point by point averaging algorithm based on work done in openFrameworks. I also used a simple shader based blur effect available in Windows Presentation Foundation as it proved way faster than any other implementations I tested as well as writing my own box blur or gaussian blur. While the results I came up with are not as good as the official SDK implementation they are pretty close and don't require detected skeletons. (The project is open source, so you can check out my implementation here: https://github.com/transcendingdigital/MFDetroit2013_Kinect_GreenScreen_PhotoKiosk) Not requiring skeleton detection is a HUGE factor when groups of users are posing for photos.

How I think Microsoft's Background Removal API Works

The unfortunate part of the Microsoft Background Removal API is that it is closed source; so people like me who have been working on the same problem are interested to know exactly what is going on at a low level. I wish they would release a paper on the technique being used. Based on my own work there are a few things I am going to guess Microsoft is doing.


Project Information URL: http://www.technogumbo.com/2013/09/Analysis-of-the-Kinect-For-Windows-SDK-1.8-Background-Removal-API/




Contact Information:

The Discussion

  • User profile image
    Ellen C Allen

    I have no idea what you are doing Charles, but this is very interesting and just from observation of the 3 photos it seems like the center pic, Kinect SDK 1.8 gives a more realistic photo, not carrying over white lines around the photo object. What you do and understand is amazing to me!

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.