A Bridge not to far... The Kinect Common Bridge get face tracking and voice recognition.

We last mentioned the Kinect Common Bridge here, Kids, Kinect, Cinder and some C++ too... Meet the Kinect Common Bridge. Today we return to it, with a fresh new update...

Kinect Common Bridge adds Face Tracking and Voice recognition!

The newest release of Kinect Common Bridge makes it even easier to track faces and recognize speech in your C++ applications with Kinect for Windows.

This is the first update to the open source Kinect Common Bridge (KCB) released recently by MS Open Tech to make it simple to integrate Kinect for Windows scenarios and experiences in creative software development. The openFrameworks and Cinder communities have already adopted the Kinect Common Bridge. If you have been using either framework and experimented with KCB, you will find yourself right at home with its added capabilities. In the spirit of “focusing on the cool stuff” that motivates creative developers, starting the sensor and displaying a simple video treatment with face tracking can now be achieved in less than 10 lines of code! Incorporating Kinect for Windows magic in software experiences couldn’t be any easier…

image

Kinect for Windows SDK comes with support for C++ and C# development. In addition to the Kinect for Windows SDK, you can download the Kinect for Windows Developer Toolkit that offers precious guidance and samples to get started coding with Kinect for Windows (both are a prerequisite for KCB use).

The Kinect Common Bridge then aggregates the various helper libraries available in the Kinect for Windows Developer Toolkit samples, and exposes them through a single C++ API adapted to creative development.

...

MSOpenTech / KinectCommonBridge

Introduction

Kinect Common Bridge is a complement to the Kinect for Windows SDK that makes it easy to integrate Kinect scenarios in creative development libraries and toolkits.

Why KCB?

When working with the openFrameworks and Cinder community members, it was evident that they needed something similar to the managed APIs but for C++. The graphics libraries they use are written entirely in native C++ for “down to the bare metal” performance to accomplish their craft. As for the functionality, they wanted something lightweight to keep the extensions to their libraries as lightweight as possible. If you are not familiar with these libraries or any type of game development model, they do not have a typical application design pattern. They need to run as fast as possible to run simulations, update positions of objects, and then render those on screen either as fast as possible or locked in sync with the refresh of the display. This can run at typical 60 frames per second (fps) and as high as the CPU/GPU can handle.

Many familiar with Kinect know the maximum frame rate is 30fps. Using an event based model doesn’t work well for this type of development since it needs to grab the frame of data when it wants, regardless of what Kinect is doing and if it isn’t there, it will catch it next time around. It cannot block the thread that does this update/query cycle.

Taking a look at the common use case scenarios, the common tasks when working with the Kinect for Windows SDK and the device are:

  1. Select a sensor
  2. Get the color/IR, depth, and skeleton data from it.

That was the goal of KCB: allow any framework that is capable of loading the DLL direct access to the data.

Requirements

The Hardware and Software below are required to build the library:

Additionally, to take advantage of the face tracking and speech recognition capabilities you need to install:

  • Speech Server SDK: It is available at http://www.microsoft.com/en-us/download/details.aspx?id=27226. Note that depending on the OS version and target platform that you are building for, you may need to have either x86, or x64, or both on your machine.

  • Kinect for Windows Developer Toolkit: It is available at http://go.microsoft.com/fwlink/?LinkID=323589 and is necessary for face tracking functionality. After the installation, make sure that the KINECT_TOOLKIT_DIR environment variable is set. Usually its value will be something like C:\Program Files\Microsoft SDKs\Kinect\Developer Toolkit v1.8.0. Tip: reboot your machine after installation even if Windows does not prompt you. Environment variables may not be updated until you do so, causing build errors.

Getting Started

...

More advanced functionality: face tracking and voice recognition

KCB has additional support for more advanced features of the sensor such as face tracking and voice recognition. Check out the samples folder for working code that illustrates how to get up and running quickly.

KCB builds with both face tracking and voice recognition enabled. To disable these items remove the following preprocessor defines from the C++ preprocessor properties of the KinectCommonBridge project:

...

Additional Resources

Tags:

Follow the Discussion

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.