Custom Gestures, Kinect for Windows v2 and the Visual Gesture Builder


Recently on the Kinect for Windows Developer Blog Ben Lower and Angela Hillier provide over two hours of detailed training and information about the new Kinect for Windows v2 SDK Visual Gesture Builder. Make sure you also check out the VBG whitepaper, "Visual Gesture Builder: A Data-Driven Solution to Gesture Detection"

Custom Gestures End to End with Kinect and Visual Gesture Builder

In this video (part 1 of 2) we walk thru end-to-end how to create custom Kinect gestures using Visual Gesture Builder.  We discuss why you would want to use a machine learning approach to custom gesture creation and then walk through in depth all the various components of creating the gesture: recording and tagging clips, previewing the gesture, and how to call the created gesture from code.

Detailed timeline:

[00:01:32] - Overview of the video
[00:02:23] - Detection:  Heuristic vs. Machine Learning
[00:05:43] - Kinect Studio (Recordable Data Sources and Recording Clips)
[00:10:20] - How & Why to Record in Raw Format
[00:16:51] - Converting Raw Clips with KSConvert.exe
[00:21:03] - Recordable Data Sources
[00:23:17] - Introduction to Visual Gesture Builder
[00:25:30] - Creating a New Gesture Using the Wizard
[00:35:15] - Adding & Tagging Clips
[00:45:34] - Building the Gesture Database File
[00:46:10] - Explanation of Build Steps & Process
[00:52:50] - Testing the Custom Gesture
[00:56:34] - Live Preview of Custom Gesture
[00:58:40] - Summary of Visual Gesture Builder
[00:59:40] - Visual Gesture Builder Process Flow Chart
[01:01:35] - Developer Sample:  Discrete Gesture Basics
[01:04:03] - Code Walk-thru:  Discrete Gesture Basics
[01:17:54] - Kinect Resources

Resources mentioned:

Project Information URL:

Custom Gestures End to End with Kinect and Visual Gesture Builder (part 2)

Part 2 gets into continuous gestures and how to create them.  Further, we teach you how to save time in your tagging by making use of VGB's auto-tagging feature.  Finally, there are some overall tips & tricks shared.

Detailed timeline:

[00:00:30] - Overview of the video
[00:02:06] - Differences Between Discrete and Continuous Gestures
[00:04:37] - Demo:  Creating Continuous Gestures
[00:05:07] - Live Preview of Steering Gesture
[00:08:40] - Explanation of VGB Solution and Projects
[00:11:00] - Explanation of Building at Solution vs. Project Level
[00:12:31] - Sit Progress Gesture: Explanation and How-To
[00:18:40] - Tagging the Progress Gesture
[00:21:00] - Introduction to Auto-Tagging
[00:26:20] - Importance of Renaming Kinect Studio Clips
[00:28:28] - Auto-Tagging Continuous Gestures (Using Discrete Gestures)
[00:37:50] - Auto-Tagging Discrete Gestures
[00:44:20] - Verifying and Tweaking Results from Auto-Tagging
[00:48:08] - Training Tips and Techniques
[00:54:05] - Verification of Clips to Ensure Body Tracking Working
[00:58:47] - Kinect Resources

Project Information URL:

Contact Information:

The Discussion

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.