KinectEx

Today's project comes via Rene Schulte recent post, Beyond Kinect for Windows v2 - KinectEx and is a project that contains a great deal of functionality that looks like could help most Kinect projects...

KinectEx/KinectEx

KinectEx is a set of libraries and utilities that add significant capability to Microsoft's version 2 Kinect for Windows sensor and SDK. It is still under active development and contributions / pull requests are encouraged.

Main Features

Kinect DVR

DVR as in Digital Video Recorder, though this is a bit of a misnomer. This feature adds the ability to record and playback Body, Color, Depth and Infrared frames without the need for an external service. It is similar to Kinect Studio, but is written in "pure" managed .Net. As such, playback can occur on any platform that supports the .Net platform, including mobile devices (though admittedly the library has not yet been tested on anything less powerful than a Surface Pro--works fine both for recording and playback, BTW).

The other major benifit of this library over Kinect Studio is that it supports resizing and compressing color frames by using real-time JPEG compression and decompression. The result is a much smaller file. Because of the smaller size, the performance can be as good or better than storing the frames uncompressed, especially with a fast CPU. If you don't require full fidelity in your color recordings, then this provides a useful alternative to the full-frame, uncompressed recordings offered via Kinect Studio.

...

Body Joint Smoothing

The other major component currently included in the KinectEx library is joint smoothing. This capability was provided natively in later versions of Microsoft's version 1 SDk, but has yet to be added to version 2. The smoothing capability uses a pluggable architecture that can support multiple smoothing algorithms. Currently, the library supports a double-exponential smoothing algorithm similar to the version 1 implementation. It also includes an algorithm that is based on a standard Kalman filter. Both offer similar performance and are highly configurable using simple configuration options. Both integrate jitter reduction, as well as statistical smoothing.

Other Stuff

There are also some other useful utilities provided, including:

  • A number of Body extension methods that provide useful measurements like angles between bones, distance between joints, etc.
  • Named "bones" (in addition to joints) as natively trackable and measurable entities.
  • Helper classes to make it easy to get WriteableBitmap representations of Color, Depth, and Infrared frames (both live and during replay).
  • A Body extension that makes it simple to get a WriteableBitmap representation of a body/joint skeleton (or overlay one on top of an existing WriteableBitmap).
  • A fully unsealed and serializable interface and class structure that mimics and extends the sealed SDK structs. These provide methods that make it nearly transparent to transition between these structures.
  • Json.NET filters and type converters that make it super-simple to serialize and deserialize bodies to JSON.

Getting Started

There is not yet a NuGet distribution for KinectEx. Nor yet is there extensive "getting started" documentation. Class library documentation is fairly robust (available at http://kinectex.github.io/KinectEx/html/index.htm). And more documentation is certainly on the backlog, as well.

For now, though, getting started likely means cloning the repository and looking closely at the sample code provided. There are four demo applications--WPF and Windows Store representations of a Recorder application (which includes sample code to demonstrate smoothing) and a Replay application. Documentation in these samples could be better, but hopefully any gaps here will be filled with the self-documentation inherent in code. All needed code is embedded in the *.xaml.cs files.

...

Project Information URL: https://github.com/KinectEx/KinectEx

Project Source URL: https://github.com/KinectEx/KinectEx

Some snaps from the Doc's that will give you an idea about all that this project can do for you and your Kinect projects...

KinectEx Namespace

BodyExtensions
Contains a number of helpful extensions to Kinect Body (and KinectEx IBody) class. Included are methods to get individual bones (really start and end joints), find angles between bones (whether connected by a common joint or not), and to find the distance between joints. 

BodyFrameExtensions
Contains extensions to the Kinect SDK BodyFrame class which enable use of KinectEx functionality using familiar KinectSDK patterns and syntax. 

BodyImageExtension
Contains extensions to the Kinect Body (and KinectEx IBody) class that makes it easy to retrieve a bitmap representation of one or more "skeletons." 

BoneTypeEx
A class that contains values for each "bone" in a Kinect Body. The class can be used statically in much the same way as an enum. However, each value contains useful information about the bone. The class also allows for easy enumeration of bones. 

ColorFrameBitmap
A class that makes it easy to create and update a WriteableBitmap from a Kinect SDK ColorFrame. 

CustomBody
This class effectively mimics the Kinect SDK's Body struct, but does so with a class that can be serialized and is not sealed. These capabilities are needed to support both the smoothing / filtering and the DVR functions of the KinectEx library. 

CustomJoint
This class effectively mimics the Kinect SDK's Joint struct, but does so with a class that can be serialized and is not sealed. These capabilities are needed to support both the smoothing / filtering and the DVR functions of the KinectEx library. 

CustomJointOrientation
This class effectively mimics the Kinect SDK's Joint struct, but does so with a class that can be serialized and is not sealed. These capabilities are needed to support both the smoothing / filtering and the DVR functions of the KinectEx library. 

DepthFrameBitmap
A class that makes it easy to create and update a WriteableBitmap from a Kinect SDK DepthFrame. 

InfraredFrameBitmap
A class that makes it easy to create and update a WriteableBitmap from a Kinect SDK InfraredFrame. 

JointTypeEx
A class that contains values for each "joint" in a Kinect Body. The class can be used statically in much the same way as an enum. However, each value contains useful information about the joint. The class also allows for easy enumeration of joints. Can be used interchangeable with the JointType enum. 

KinectBody
This class just wraps the Kinect SDK's Joint struct, but does so with a class that implements the common IJoint interface. This makes it possible to use a standard Kinect SDK object with many of the utility functions provided by the KinectEx library. 

KinectJoint
This class just wraps the Kinect SDK's Joint struct, but does so with a class that implements the common IJoint interface. This makes it possible to use a standard Kinect SDK object with many of the utility functions provided by the KinectEx library. 

KinectJointOrientation
This class just wraps the Kinect SDK's JointOrientation struct, but does so with a class that implements the common IJointOrientation interface. This makes it possible to use a standard Kinect SDK object with many of the utility functions provided by the KinectEx library.

KinectEx.DVR

JpegColorCodec
An IColorCodec that performs compresses bitmaps using the JPEG standard. Choosing this codec when creating a KinectRecorder effectively saves color frames as an MJPEG. 

KinectRecorder
This class is one of two primary programmatic interfaces into the KinectEx.DVR subsystem. Created to enable recording of frames to a Stream. 

KinectReplay
This class is one of two primary programmatic interfaces into the KinectEx.DVR subsystem. Created to enable playback of frames from a Stream. 

RawColorCodec
An IColorCodec that performs no image compression. By default, this will just encode and decode the raw bitmap data. However, if the OutputWidth and/or OutputHeight are changed, it will resize an encoded bitmap accordingly. 

ReplayBodyFrame
A recordable / replayable version of a BodyFrame. 

ReplayColorFrame
A recordable / replayable version of a ColorFrame. 

ReplayDepthFrame
A recordable / replayable version of a DepthFrame. 

ReplayFrame
Base class for all forms of recordable / replayable frames. 

ReplayFrameArrivedEventArgs<T>
Generic EventArgs type used to pass the specific type of ReplayFrame to a listener during playback when that frame should be processed or displayed. 

ReplayInfraredFrame
A recordable / replayable version of an InfraredFrame.

KinectEx.Json

BoneTypeExConverter
Json.Net converter that serializes a BoneTypeEx object as a single string and deserializes it to the appropriate static instance. 

JointTypeExConverter
Json.Net converter that serializes a JointTypeEx object as a single string and deserializes it to the appropriate static instance. 

KinectTypeBinder
A SerializationBinder that handles the tricky issue of cross-platform JSON serialization when storing and retrieving objects that come from different assemblies and namespaces depending on the platform. In this case, the Kinect SDK classes need to be able to be easily converted to and from WindowsPreview.Kinect, Microsoft.Kinect, or KinectEx.KinectSDK.

KinectEx.Smoothing

ExponentialJoint
A class representing a CustomJoint who's position values are "smoothed" using a Double Exponential filtering algorithm. 

ExponentialJointOrientation
A class representing a CustomJointOrientation who's values are "smoothed" using a Double Exponential filtering algorithm. 

ExponentialSmoother
A smoothing "strategy" that, when applied to a SmoothedBody will "smooth" the joints using a Double Exponential filtering algorithm. 

ExponentialSmoothingParameters
Parameters used to control the smoothing of an ExponentialSmoother. 

KalmanJoint
A class representing a CustomJoint who's position values are "smoothed" using a Kalman-like filtering algorithm. 

KalmanJointOrientation
A class representing a CustomJoint who's position values are "smoothed" using a Kalman-like filtering algorithm. 

KalmanSmoother
A smoothing "strategy" that, when applied to a SmoothedBody will "smooth" the joints using a Kalman-like filtering algorithm. 

KalmanSmoothingParameters
Contains member properties used to configure a KalmanSmooter. 

SmoothedBody<T>
A generic class represents a CustomBody that implements the smoothing "strategy" represented by the specified ISmoother. 

SmoothedBodyList<T>
A specialized list of SmoothedBody objects.




Tags:

Follow the discussion

  • Oops, something didn't work.

    Getting subscription
    Subscribe to this conversation
    Unsubscribing
    Subscribing

Comments closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.