Saving Kinect Streams - Skeleton Stream to BVH Converter and Stream Saver Application

Today we're going to highlight two different projects, both relating to Streaming...

The first project by Sebastian is a simpler project that, well, the title really says it all.

Kinect Skeleton Stream to BVH Converter

Hi ,
during a project at my University (Karlsruhe Institute of Technology) I created this code which allows you to save the Kinect Skeleton Stream in the BVH Data format. The Code is written in C# and the Kinect SDK 1.7 is used.

Regards,
Sebastian

Project Information URL: https://kinect2bvh.codeplex.com/

Project Source URL: https://kinect2bvh.codeplex.com/SourceControl/latest

 

The second project, by Elham Dolatabadi of The AI & Robotics Research Team (AIRR), Toronto Rehabilitation Institute is a little more complex...

Kinect Stream Saver Application

The Kinect Stream Application allows users to display and store the Kinect streams. This application is developed based on a sample called "Kinect Explorer - D2D C " developed by Microsoft corporation. (Compatible with SDK 1.7)

Noncommercial — You may not use this application for commercial purposes. 
>> © The AI & Robotics Research Team (AIRR), Toronto Rehabilitation Institute.
Prepared by Elham Dolatabadi,

The Kinect Stream Saver application is meant to aid research in the general area of mobility monitoring. The AI & Robotics Research Team (AIRR) within the Toronto Rehabilitation Institute (TRI) holds copyright of the dataset.

Implementation:

The Kinect Stream Saver application is based on the KinectExplorer-D2D (for C++) sample code.

Modifications to the sample code are explained below.

- Two new classes were added.

  • StreamSaver:

This class has methods that store stream data (color, depth, skeleton) to a dynamic FIFO buffer as the data is received from the Kinect device, and separately write the data from the FIFO buffer to output files as described above. The disk writes are done in a separate thread to avoid degradation of frame processing speed during data collection.

  • SaverViewer:

This class implements a fourth GUI tab in the KinectExplorer application. The tab is labeled "Recording" and it allows users to specify which data streams should be saved to disk.

- A few classes in the KinectExplorer sample code were modified. The changes were kept to minimum, but   were necessary to invoke the functionality of the new classes.

  • KinectWindow
  • NuiColorStream
  • NuiDepthStream
  • NuiSkeletonStream

- A new GUI "Recording" tab was designed and included in the application resources. This included modifications to the following files.

  • KinectExplorer.rc
  • resource.h

Instructions for use:

To collect data with the Kinect Stream Saver:
Run KinectExplorer-D2D.exe:

  • In the lower right corner of the window navigate to the 4th tab, labeled Recording.
  • Enter the Local Disk name and Folder Name if desired. Otherwise, data will be collected at C:\KinectData
  • Choose which data streams to collect.
  • Press Capture to begin data collection.
  • Press Stop to end data collection.

The collected data will appear in Local Disk:\KinectData\Folder Name.

  • LiTimeStamp.csv :

This file has fields: counter#, frame#, timestamp

    • The counter starts at 1 when the Capture button is pressed and data collection begins.
    • The frame number is from the Kinect API; it starts when KinectExplorer-D2D.exe is launched.
    • The timestamp is in milliseconds, and is useful for recording frame rate. (At present data is collected at approximately 30 frames/second, so frames are approx 33 msecs apart.)
  • Color images are saved as .bmp files, on each frame.
  • Depth images* are saves either

. as .bmp files on each frame.
. as "raw" data stored in a .csv file.

  • Skeleton data:

The program can track up to six skeletons at once. For the first four, all joints are tracked.

For the last two, only the center of mass is tracked.

Saving The Joint Positions: Six data files are created, named Joint_Position_1.csv through Joint_Position_6.csv, whether six skeletons were tracked or not.

Saving The Joint Orientations: Six data files are created, named Joint_Orientation_1.csv through Joint_Orientation_6.csv, whether six skeletons were tracked or not.

Twenty-one rows are written to each skeleton data file for each frame. These 21 rows store data for the center of mass, followed by the 20 skeleton joints in the following order.

CoM Position
HIP_CENTER
SPINE
SHOULDER_CENTER
HEAD
SHOULDER_LEFT
ELBOW_LEFT
WRIST_LEFT
HAND_LEFT
SHOULDER_RIGHT
ELBOW_RIGHT
WRIST_RIGHT
HAND_RIGHT
HIP_LEFT
KNEE_LEFT
ANKLE_LEFT
FOOT_LEFT
HIP_RIGHT
KNEE_RIGHT
ANKLE_RIGHT
FOOT_RIGHT

--- "JointPosition....csv" File ---
Each row contains following fields:
- position (x, y, z)
- frame number (in CoM row); joint tracking state (in joint rows)*
* Tracking states: 0=joint is not tracked; 1=joint is inferred; 2=joint is tracked

--- "JointOrientation....csv" File ---
Each row contains following fields:
- hierarchical joint angle as a quaternion (w, x, y, z),
- angle status*
* Angle status:
- in joint rows: 0=successful, any other value implies angle computation failed and the angle has been set to the Identity quaternion (1,0,0,0).
- in CoM rows: Not applicable. Angle status is set to S_FALSE=1.

Caveats:

  • If fewer than 6 skeletons are tracked, the data may appear in any of the 6 files. Check the file size to determine which files store skeleton data. The unused files will contain only frame numbers and zeros for all positions in all frames.
  • If a user that is being tracked leaves the field of view and later returns, the Kinect does not know it is the same person, so they will be assigned a new skeleton id. This can mean that the person's data appears in more than one data file.

Project Information URL: https://kinectstreamsaver.codeplex.com/

Project Source URL: https://kinectstreamsaver.codeplex.com/SourceControl/latest

Tags:

Follow the Discussion

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.