Seems fitting to jump forward a couple years, and highlight another such project, yet one that's one step (or two) beyond. Light Saber duels anyone? Hulk Smash?

And there's source of course!

Slides and Demos for Programming With Kinect for Windows Talk, and Introducing JediSmash

I have been playing a lot with the Kinect for Windows this year, and gave presentations on it at LRTechFest and Cowtown CodeCamp recently. As promised, you can find the slides and demos here.

I also showed a fun WPF application that combines multiple Kinect capabilities like Color, Depth, Skeletal tracking and speech recognition. It’s called JediSmash.

I also got a chance to show the Kinect Replay application that I built that lets you record and replay Kinect sessions without having the actual Kinect hardware. I’ll blog about it soon. I am excited about the version 1.7 of the SDK that came out this week, and am planning to talk about the new features as I do more Kinect related talks in user groups/conferences in Texas and surrounding areas this year. If you lead a user group and would like me to present, feel free to get in touch

Project Information URL: http://dotnetsurfers.com/blog/2013/03/17/slides-and-demos-for-programming-with-kinect-for-windows-talk/

Project Source URL: JediSmash

You know what? We're going to make this a two for one day! Here's another great post by Latish...

Kinect.Replay: Record and Replay Your Kinect Sessions

One of the limitations of programming with Kinect is that you always need the Kinect hardware plugged in during development time. You can use Kinect Studio to record and replay sessions (it gets tiring after a while to get up repeatedly in front of the Kinect and debug at the same time, and your spouse/kids/pets start looking at you funny), but you still need the Kinect hardware to replay a Kinect Studio session. Being hardware independent can be useful in scenarios like programming while on the move (Most of my side projects are done in coffee shops), or running integration tests on a server. I created Kinect.Replay as an experiment to see if I can remove the dependency on the hardware during some parts of development. The core of the code is based off the work done by David Catuhe in Kinect Toolbox. My initial approach was to enhance Kinect Toolbox, but the direction I was taking soon started to look like I was butchering the code, and I ended up creating a separate project for my needs. Some of the enhancements I have done include storing ColorToDepthRelationalParameters to recreate CoordinateMapper during replay, and also support for audio recording/replay. You can see a small demo of what the using Kinect.Recorder looks like in the video below.

Project Information URL: http://dotnetsurfers.com/blog/2013/03/19/kinect-dot-replay-record-and-replay-your-kinect-sessions/

Project Source URL: Kinect.Replay

Contact Information:

• Blog: http://dotnetsurfers.com/blog
• Twitter: @Latish

• [00:26] The latest Kinect for Windows [v1.7] SDK is here
• [01:38] Windows Azure: New Hadoop service + HTML5/JS (CORS), PhoneGap, Mercurial and Dropbox support (Scott Guthrie), Getting Started with the Mobile Services HTML Client (Nick Harris)
• [02:43] SignalR eBook (campusMVP.Net, Jose M Aguilar) [Found via: The Morning Brew #1316]
• [03:34] Modeling Neuron Behavior in C# (James McCaffrey)
• [04:59] Aviary Launches Windows 8 SDK with 6 Partners
• [06:00] dx.h - A modern C++ library for DirectX programming (Kenny Kerrca)
• [06:52] An Introduction to Meteor (Stephen Walther)
• [07:54] From a side project to Windows 8 Store featured app - the Interference story (Sara Allison)

Picks of the Week!

• Dan's Pick of the Week:[08:55] Treat Your Kids Like a Team of Agile Software Developers and Lower Your Family Stress (Whitson Gordon)
• Clint's Pick of the Week:[10:50] Kodu Challenge (Jennifer Ritzinger)

Hot Links:

• Latest SDK announcement
• Download the SDK 
• Kinect for Windows key features
• See Latest SDK in action

Kinect for Windows announces new version of SDK coming March 18

On Monday, developers will be able to download the SDK, developer toolkit, and the new and improved Human Interface Guidelines (HIG) from our website. In the meantime, here’s a sneak peek:

Kinect Interactions give businesses and developers the tools to create intuitive, smooth, and polished applications that are ergonomic and intelligently based on the way people naturally move and gesture. The interactions include push-to-press buttons, grip-to-pan capabilities, and support for smart ways to accommodate multiple users and two-person interactions. These new tools are based on thousands of hours of research, development, and testing with a broad and diverse group of people. We wanted to save businesses and developers hours of development time while making it easier for them to create gesture-based experiences that are highly consistent from application to application and utterly simple for end users. With Kinect Interactions, businesses can more quickly develop customized, differentiated solutions that address important business needs and attract, engage, and delight their customers.

...

Kinect Fusion is one of the most affordable tools available today for creating accurate 3-D renderings of people and objects. Kinect Fusion fuses together multiple snapshots from the Kinect for Windows sensor to create accurate, full, 3-D models. Developers can move a Kinect for Windows sensor around a person, object, or environment and “paint” a 3-D image of the person or thing in real time. These 3-D images can then be used to enhance countless real-world scenarios, including augmented reality, 3-D printing, interior and industrial design, and body scanning for things such as improved clothes shopping experiences and better-fitting orthotics. Kinect Fusion is something many of our partners have been asking for and we’re thrilled to be delivering it now.

The updated SDK also includes an enhanced developer toolkit and additional developer resources, including:

• OpenCV and MATLAB Samples to allow developers to build advanced Kinect-enabled applications while using industry standards.
• Kinect for Windows Code Samples on CodePlex

..."

Exciting times isn't it?

Update: The SDK is now available for download...

Key Links

• Latest SDK announcement
• Download the SDK
• Kinect for Windows
• Kinect for Windows key features
• See Latest SDK in action
• Hear what people have been saying

Kinect for Windows Samples

The Kinect for Windows (K4W) team release samples to show how to build applications and experiences using K4W.  Our “Basics” samples are designed to quickly show a particular feature.  Other samples are more robust and can serve as a template for building an app.  You will still want to download and install our SDK and Toolkit for the full developer experience but the code here is easy to browse and you can submit feedback about the samples right here on the site.

...

List of Samples

This is the list of all samples included in the latest Developer Toolkit release. The table lists the name of the sample, in which languages it’s available and what technologies and additional SDKs are used.  You can browse each sample’s source code by using the “Source Code” tab above.

Project Information URL: http://kinectforwindows.codeplex.com/

Project Source URL: http://kinectforwindows.codeplex.com/

Contact Information:

• Blog: Kinect for Windows Developer Blog 

• [0:42] March CTP (“CTP4”) of Visual Studio Update 2 now available (Charles Sterling)
• [2:07] Now Available: Office Developer Tools for Visual Studio 2012 (S. Somasegar)
• [3:33] Visual Studio Tools for Git 0.8.0.0 (Andy Lewis)
• [4:49] Announcing a New Preview of the LightSwitch HTML Client
• [6:07] Getting Started with LightSwitch in Visual Studio 2012 Update 2 CTP4 (Beth Massi)
• [6:40] MetroAppSite: Free, Open Source Metro-Style Website Templates for Your Windows Store Apps (Aaron Stannard)
• [7:35] Recording of Webinar 'Applied MVVM in Windows 8 apps' by Gill Cleeren (Gill Cleeren)
• [9:01] Advanced MVVM concepts using Contoso Cookbook: An analysis of a complete MVVM application: Part 1 (Gill Cleeren)
• [9:44] Easy Access to Kinect for Windows Sample Code (Ben Lower)
• [10:34] Interpolation Tricks

Picks of the Week!

• Dan's Pick of the Week: [12:06] Unity Windows Store apps open beta, and Channel 9 Live event!
• Brian's Pick of the Week: [12:52] Panoramic Camera Head

The winners of the 2012 Engadget Awards -- Readers' Choice - Peripheral of the Year: Microsoft Kinect (2012)

Today, we are pleased to announce that Kinect for Windows was recognized by Engadget for a Reader’s Choice Award!

Project Information URL: http://www.engadget.com/2013/02/20/winners-2012-engadget-awards-readers-choice/

This is all your fault! All of you creating awesome games, environments, interactions, educational systems and so much more, pushing what started as a game peripheral into a whole new world beyond anything originally imagined.

You rock!

Admin Note: Just a quick note to let you know we're scaling back the number of Kinect Gallery posts for a bit. The current plan is to post once or twice a week. If we get a news spike, then we'll go back to once every business day.

moveIT

moveIT is an add-in application for Autodesk Inventor that enables the posing and positioning of a human body model, using Microsoft Kinect sensor and movements of your own body.

The moveIT add-in enables the posing of a human body model with your own movements. For example, if you lift a hand or bend an elbow, the pose of the human body model open in Autodesk Inventor updates accordingly.

Key features:

• All major joints are supported: head, torso, arms, elbows, shoulders, hips, knees and legs
• Ability to control not only the pose but the position of the model as well: if you jump or make steps in any direction, the position of the human body model in Autodesk Inventor updates accordingly
• Ability to adjust size and proportions of the human body model to better fit specific needs
• Ability to interact with the virtual environment, 'activating' and 'deactivating' blocks with a touch

Project Information URL: http://amcbridge.com/?q=MoveIt

Project Download URL: http://amcbridge.com/?q=MoveIt

Kineed For Speed – Play Need For Speed (Or Other Racing Games) with Kinect

Kineed For Speed = Kinect + Need For Speed

It can be used to play PC version of Need For Speed (or other racing games) with Kinect..Just like playing Kinect Joy Ride on Xbox..

How to use
Simply Run the KineedForSpeed.exe and a racing game at the same time.. You can play the game with Kinect..

How to gain control:
1. hold hands in gesture like holding a virtual wheel to gain the control.
2. lower hands to release the control.

How to drive:
1. Turn left/right by turning the virtual wheel left/right.
2. Speed up by putting the virtual wheel forward.
3. Slow down by retrieving back the virtual wheel.

Configurable key mappings:
1. Left hand up to trigger Esc key event, which will pause the game.
2. Right hand up to trigger Enter key event, which will reset the car.

You can also set other keys to make it work with other racing games..

Project Information URL: http://tangochen.com/blog/?p=564

Project Download URL: http://tangochen.com/blog/wp-content/plugins/download-monitor/download.php?id=15

Contact Information:

• Blog: http://tangochen.com/blog
• Twitter: @tangochen

Today Brian Peek announces that the latest Coding4Fun Kinect Service now works for WinRT clients!

Coding4Fun Kinect Service v1.6

WinRT clients! There are native Windows Runtime Components, so use the proper architecture(s) for your project: x86, x64 and/or ARM.

This is the first version which includes client services for Windows Store Apps, written as a C++ Windows Runtime Component, with binaries available for x86, x64 and ARM.

Code samples are included in the source repository and in the download on CodePlex for XAML and JavaScript Windows Store Apps.  The rest of the code base remains pretty much identical, with the only change being the Kinect portions are now compiled against the v1.6 Kinect SDK.

What is the Coding4Fun Kinect Service?

The Coding4Fun Kinect Service allows you to stream Kinect color, depth, skeleton, and audio from one PC to another PC or a Windows Phone via sockets. Both server and client libraries are available to send and receive the data.  Please review the included WPF, Windows 8, and Windows Phone samples for a quick explanation of how to use the libraries.  Also check out the Documentation tab for more information and sample usage.

This project requires the Kinect for Windows SDK v1.6.  If you want to learn how to use the Kinect for Windows SDK, head over to the Channel 9 and view the Quickstart series.

Project Information URL: http://kinectservice.codeplex.com/

Project Download URL: http://kinectservice.codeplex.com/releases/

Project Source URL: http://kinectservice.codeplex.com/SourceControl/BrowseLatest

Transforming Computer Interaction with Natural User Interfaces

Microsoft Research is only scratching the surface when it comes to developing new ways users can naturally interact with computers, or, with the aid of small natural user interface devices, interact with the world around them, whether it's the world of gaming, medicine, or everyday computing.

Project Information URL: http://www.youtube.com/watch?v=i3Up4k1K4QE&list=UUCb9_Kn8F_Opb3UCGm-lILQ&index=2

Found via: New Microsoft Research Trailer Shows Mostly Kinect-Related Projects

The Door is Open for You to Kinect

I’m not sure that many of you know this, but Kinect for Xbox 360 holds the Guinness Book of World Records for the fastest selling consumer electronic device in history.  Since its launch users around the world have been using Kinect in creative new ways.  We’ve dubbed this the “Kinect Effect” here at Microsoft.  Kinect for Windows has taken this even further by opening a doorway into seeing and interacting with the world that many of us thought was solely for the movies.  Just this week the technology behind Kinect for Windows – including Windows Embedded – is on display at New York Fashion Week in the form of Swivel, a virtual dressing room from FaceCake that gives people the chance to try on clothes without changing. What a concept.

Today, another member of The Embedded Ninja clan, Eric Kamont, shares his observations and examples of leveraging Kinect as part Intelligent Systems.  Kinect and enjoy!

Project Information URL: http://blogs.msdn.com/b/windows-embedded/archive/2012/09/11/the-door-is-open-for-you-to-kinect.aspx

Kinect for life

This week we had an event showcasing  medical technology innovation in partnership with Kingston University, the University of Surrey, Brunel University and Microsoft.

Given the revolutionary advances made possible with Microsoft’s Kinect for Windows, Medical professionals and researchers are exploring how computer vision and natural user interfaces can enhance healthcare.

Confirmed sessions:

• Fall detection system, Dr Dimitrios Makris, Kingston University
• Facial expression recognition from 3D data, Dr Hongying Meng, Brunel University
• Controlling a smart home, Dr Francisco Florez Revuelta, Kingston University
• Concept to Commercialisation – A strategy for business innovation 2011- 2015– Graham Worsley, Lead Technologist in the Assisted Living Innovation Platform, Technology Strategy Board
• Kinect for Medical and Non-gaming applications: developments at the University of Surrey - Dr Kevin Wells, University of Surrey

Confirmed speakers:

• Graham Worsley, Technology Strategy Board
• Prof Malcolm Sperrin, Royal Berkshire Hospital
• Dr Dimitrios Makris, Kingston University
• Dave Brown, Microsoft
• Prof Paolo Remagnino, Kingston University
• Dr Kevin Wells, University of Surrey
• Dr Hongying Meng, Brunel University
• Dr Francisco Florez Revuelta, Kingston University
• Tim Craig, Smart Care UK

Kinect use examples

...

This is a preview into a technology that will be used in the future for example to perform remote surgery or to send robots to work in dangerous areas.

More information on our Kinect + Robot project can be found here:http://robosavvy.com/forum/viewtopic.php?t=8026

Microsoft Robotics Studio

There is currently no standard regarding the control of Robots.  

Most small robots, use a low power microcontroller similar to Arduino. This is something like a computer, but much less powerful, however suitable to communicate with sensors, control motors, recharge batteries, etc.

There is a huge variety of these microcontrollers, and even when the more usual types are used, the robot manufacturers usually create their own software to operate the robots.

This causes situations where for example when a program is designed for a given robot, it needs to be completely rewritten if another brand of robot is to be used, even if they are nearly identical at an hardware level.

This where Microsoft Robotics Studio steps in and closes that gap. Robot manufacturers, or the users themselves can design small software modules for each robot, that act as a translator between MRDS and the control system for the robots.

This means that in MSRDS a command, for example to make a humanoid robot step forward, is identical across several brands of robots.

With MSRDS, Robots can now talk to each other, talk to sensors from different manufacturers or be supervised by our own master process (hypervisor) that makes sure everything is working as expected.   MSRDS also enables interoperability with complex functionalities hosted on the PC such as Speech Recognition.

Another advantage of MRDS is that it is a tool accessible to wide range of users, regardless of their expertise. Beginners can build Robot behaviours using Visual Programming Language and Advanced users can work with textual programming (any .Net language) to make the most out of MSRDS.

...

Project Information URL: http://blogs.msdn.com/b/uk_faculty_connection/archive/2012/10/05/kinect-for-life.aspx, http://robosavvy.com/forum/viewtopic.php?t=8026

Project Download URL: http://robosavvy.com/forum/viewtopic.php?t=8026

Project Source URL: http://robosavvy.com/forum/viewtopic.php?t=8026

Contact Information:

• Blog: http://blogs.msdn.com/b/uk_faculty_connection/

Kinect Games v2.5 using Kinect SDK v1.6 including source code

This new release of Kinect Games fixes a couple of bugs from the last release which affected performance, the main thing being that I wasn’t correctly disposing of some unmanaged resources which slowed down the framerate. As well as fixing those bugs I have also used the new Kinect SDK v1.6 in this release which should also improve performance.

I have also added a new game to the installer called Kinect Pong, which is just a simple pong game which can be played 1 player against the computer or in 2 players mode. You simply use your right hand to control the bat; the game senses whether 1 or 2 players are standing in front of Kinect and automatically implements AI for the 2nd bat if only 1 person is playing.

There are a few keyboard controls for Kinect Pong:
F11 toggles full screen mode
UP and DOWN cursor keys move the Kinect camera angle up and down
SPACE BAR toggles the video feed on and off (turning it off will improve framerate on slow computers)
ENTER starts a new game when a game has ended
ESCAPE quits the game

Please click here to download the new all in one installer for Kinect Games version 2.5, the installer includes Kinect Angles, Kinect Time, Kinect Magic Cursor, Kinect Pong, .net 4.0 framework, XNA4 runtime and Kinect SDK v1.6.

Source Code
Click here to download Kinect Angles v2.5 source code
Click here to download Kinect Time v1.6 source code
Click here to download Kinect Pong v1.6 source code
Click here to download Kinect Magic Cursor v1.6 source code

You might notice in the Kinect Pong source code that I haven’t used classes much, that was because I was writing the Pong game for a structured programming unit I teach which doesn’t go into classes but only structures. I simply added the Kinect stuff into the game I wrote for that class, so that is why it doesn’t use classes much. I suppose if you are a beginner to XNA/Kinect programming this may make the code easier to follow, or not as the case may be

Project Information URL: http://drenton72.wordpress.com/2012/10/15/kinect-games-v2-5-using-kinect-sdk-v1-6-including-source-code/

Project Download URL: https://dl.dropbox.com/u/51033091/xGames/KinectGamesv2_5.exe

Project Source URL: Kinect Angles v2.5 source code, Kinect Time v1.6 source code, Kinect Pong v1.6 source code, Kinect Magic Cursor v1.6 source code

Contact Information:

• Blog: http://drenton72.wordpress.com
• Twitter: @drenton72

Virtual Events Sample - KINECT for Windows and Windows 8 Store App with Avatar

This sample app is composed by:

- A Server application (WPF App) which is controlling a KINECT for Windows and broadcasting the data to the network so it it consumed by 'n' clients.

- A remote Client app (Windows 8 Store C++ & DirectX app) which renders an 3D AVATAR showing every movement made by the guy who is in front of the KINECT for Windows.  

Requirements of the Sample

Requirements:

• KINECT for Windows SDK 1.5
• Visual Studio 2012
• Windows 8

Description

This is a WOW app which shows topics like Connected devices, Windows 8 Store apps and C++ 3D development.

The new Windows Store apps that are built using C++ form a key pillar in the context of modern app development. This is especially important when you want to create the best quality and highest-performing apps capable of running on tablet hardware which require a long battery life and great performance on slower processors. 

 

Running the Sample

First, you need to run the Server app (ServerUI - WPF .exe). You need to have connected a KINECT for Windows to that machine.

Then, you configure the Virtual event characteristics (name, description, speaker, speaker's photo, KINECT and an optional WebCam for background images). finally, press the START button. It will start broadcasting the data.

Second, run the Windows 8 Store App (VirtualEventsClient) in any Windows 8 machine (a tablet is convenient). Join to the virtual event published, and HAVE FUN!

The WebAPI and Azure projects are not required to just make it work. They are a convenient location system that you could upload to any server.

Related App (MY EVENTS)

This app is related to the sample app called 'MY EVENTS - Sample Modern Application with Visual Studio 2012'. This is the URL of this other sample app:http://code.msdn.microsoft.com/MY-EVENTS-Modern-Sample-9a42abc6

Project Information URL: http://code.msdn.microsoft.com/Virtual-Events-Sample-a0228a97

Project Download URL: http://code.msdn.microsoft.com/Virtual-Events-Sample-a0228a97

Project Source URL: http://code.msdn.microsoft.com/Virtual-Events-Sample-a0228a97

Note: If you have a problem building/compiling this Solution, try moving it up a folder level. On my system I ran into a build/linking error initially but when I moved the source from the ...\C#,C++\... folder up a level it compiled fine (some warnings, but no errors). Worse case, the precompiled binaries are in the zip too.

Satellites to employ Kinect to dock in space

Microsoft's motion-tracking camera and software is extending further beyond its initial gaming audience, with UK engineers planning to dock a satellite pair by using the Kinect.

Microsoft's motion-tracking camera and software is extending further beyond its initial gaming audience, with UK engineers planning to dock a satellite pair by using the Kinect.

Surrey Satellite Technology Limited (SSTL) is developing the STRaND-2 satellites that will take advantage of the Kinect's ability to provide spatial awareness in order to split and dock the pair while in orbit.

Shaun Kenyon, Project Lead at SSTL, said that the company was impressed by a previous project by MIT that used the Kinect to fly an autonomous model helicopter, leading to SSTL's consideration of the technology for space missions.

Project Information URL: http://vr-zone.com/articles/satellites-to-employ-kinect-to-dock-in-space/16063.html#ixzz1wfzEkYiL

Science & Exploration - STRaND-2 docking nanosatellite

Space innovators at the University of Surrey and SSTL are developing STRaND-2, a twin-satellite mission to test a novel in-orbit docking system based on the XBOX Kinect technology.

Similar in design to STRaND-1, the identical twin satellites will each measure 30cm (3 unit Cubesat) in length, and will utilise components from the XBOX Kinect games controller to scan the local area and provide the satellites with spatial awareness on all three axes - thus allowing them to dock.

The STRaND team sees the relatively low cost nanosatellites as intelligent "space building blocks" that could be stacked together and reconfigured to build larger modular spacecraft.

Project Information URL: http://www.sstl.co.uk/divisions/earth-observation---science/science---exploration/strand-2-building-blocks-nanosatellite

kinect-ultra

What's this?

This is a Kinect Hack that allows you to Transform yourself to Ultra Seven.

This program may freeze or crash on your computer. It may be caused by a bug, your GPU's capability, your graphic driver, or any other possibilities. I cannot tell you exactly why it happens and how you can workaround it, but I will make any information available on this site as far as known. Please come back often and check the latest information. Your cooperation and patience would be much appreciated.

Quick start

If you have Windows 7, download and install the latest Microsoft's Kinect SDK. Then download the latest kinect-ultra for Kinect SDK, extract it wherever you like, and then run kinect-ultra.exe.

To run on the OpenNI framework, kinect-ultra for OpenNI is also available. For more information about which to choose, see FAQ_en.

What's new

• (5/22/2011) Tested on Kinect SDK 1.5. Working fine.
• (4/23/2011 V1.0a) Kinect SDK version that works without OpenNI is coming up! The transformation pose is now changed to the motion of putting the Ultra Eye on instead of the Psi pose.
• (9/18/2011 V0.1f) Minor screen size related enhancements and readme fixes. Now the app runs in full screen by hitting ENTER key.
• (1/9/2011 V0.1a) First release!

Description

Run this program on Windows PC with fairly fast CPU and fairly advanced GPU (with at least OpenGL 2.0 and programmable shader capabilities). WorkingConfigurations page lists the configurations this program is reported to work on. Please let me know of your configuration by a comment on the same page.

If you have any problems, please look at FAQ page for solution. If it does not solve the problem, feel free to report them at Issues, or just come back some time later to check if your issues are resolved. Because this program does not have much exposure to other configurations yet than I have, it will take some time to become stable.

If you have any other comments, please leave them at Comments page.

This version supports the following Ultra Sever's superpowers:

Transform by the pose of putting Ultra Eye on

Wide Shot

Emerium Beam (A/B type)

Eye Slugger (throw, hold, and Ultra Knock Attack)

Time limit and flying back

Thanks to panzerNEWB for nice Eye Slugger model.

See Articles for links to online articles referencing to this project.

Project Information URL: http://code.google.com/p/kinect-ultra/

Project Download URL: http://code.google.com/p/kinect-ultra/downloads/list

Project Source URL: http://code.google.com/p/kinect-ultra/source/checkout

Kinect Calibration

This program adjusts Kinect to better see the user, this way the user doesn’t have to move back or forward (unless Kinect’s angles aren’t enough to see the user). I think this is a useful thing to have when you start your Kinect application.

The code is very simple to follow. Use Kinect’s elevation angle to change the Kinect’s “view” and track the user in sight. For each angle count how many joints Kinect is tracking and save that angle as the best if we reach a new maximum number of tracked joints.

...

You may notice two things about the code:

• First is that I’m using time to control between Kinect movements. The reason for this is that if we try to compare Kinect with the angle we set, we might not get the same value for different reasons (Kinect sensor is not 100% accurate or it couldn’t physically rotate the sensor to that exact angle).
• And second is the fact that I’m only scanning the skeleton for some angles. I don’t know if this is a Kinect’s limitation or if I did something wrong in the code but I couldn’t track any joints while Kinect was moving. So what I did was move the sensor between some angles and wait there a bit to count the bones for each one of those angles.

Known limitations:

• it only supports 1 user at this moment;
• the scanning process is not ideal.

Project Information URL: http://n0n4m3.codingcorner.net/?p=854

Project Source URL: http://n0n4m3.codingcorner.net/wp-content/uploads/2012/05/KinectCalibration.zip

Contact Information:

• Blog: http://n0n4m3.codingcorner.net
• Twitter: @l0kus

Face Tracking SDK in Kinect For Windows 1.5

After a long journey, my team at Microsoft shipped Face Tracking SDK as part of Kinect For Windows 1.5! I worked on the 3D face tracking technology (starting from the times when it was part of Avatar Kinect) and so I’d like to describe its capabilities and limitations in this post. First of all, here is the demo:

You can use the Face Tracking SDK in your program if you install Kinect for Windows Developer Toolkit 1.5. After you install it, go to the provided samples and run/build yourself “Face Tracking Visualization” C++ sample or ”Face Tracking Basics-WPF” C# sample. Off course, you need to have Kinect camera attached to your PC The face tracking engine tracks at the speed of 4-8 ms per frame depending on how powerful your PC is. It does its computations on CPU only (does not use GPU, since it may be needed to render graphics).

If you look at the 2 mentioned code samples, you can see that it is relatively easy to add face tracking capabilities to your application. You need to link with a provided lib, place 2 dlls in the global path or in the working directory of your your executable (so they can be found) and add something like this to your code (this is in C++, you can also do it in C#, see the code samples):

Project Information URL: http://nsmoly.wordpress.com/2012/05/21/face-tracking-sdk-in-kinect-for-windows-1-5/

How To Use Kinect Face Tracking SDK

Introduction

This article demonstrates how to use the Face Tracking SDK in Kinect for Windows to track human faces. It provides code samples as well as useful tips on how to call its APIs to get the most out of the face tracking engine.

Background

The Face Tracking SDK is part of Kinect For Windows Developer Toolkit and can be installed from this site. It can be used for markerless tracking of human faces with Kinect camera attached to a PC. The face tracking engine computes 3D positions of semantic facial feature points as well as a 3D head pose. The Face Tracking SDK could be used to drive virtual avatars, recognize facial expressions, Natural User Interfaces and other face related computer vision tasks.

The full API reference is available on MSDN site as part of Kinect For Windows SDK help. I worked on the development of the face tracking engine and its API and so can provide a good overview of its usage. The general overview and some usage tips can also be found in this post on my site.

Using the Face Tracking SDK in your code

You can use the Face Tracking SDK in your program if you install Kinect for Windows Developer Toolkit 1.5. After you install it, go to the provided samples and run/build yourself “Face Tracking Visualization” C++ sample or "Face Tracking Basics-WPF" C# sample. You need to have Kinect camera attached to your PC. The Face Tracking engine tracks faces at the speed of 4-8 ms per frame depending on your PC resources. It does its computations on a CPU (does not use GPU).

This picture demonstrates the results of face tracking. The yellow mask is the 3D mask fit to the face projected to the RGB frame.

...

Project Information URL: http://www.codeproject.com/Articles/394975/How-To-Use-Kinect-Face-Tracking-SDK

Contact Information:

• Blog: http://nsmoly.wordpress.com/

Human Body

This application helps children learn English. It uses Drag and Drop algorithm for Kinect. The aim of this app is to select word, drag it and drop on the correct place. Powered by natural user interface.

About

Learning English with this app is not boring, children are able to drag some word from the bottom of the screen, drag it and drop on the correct place. They are not sitting as traditional but the are moving in front of the device.

This is simple example of innovations in learning. Games could interested children and helps them growing knowledge.

Features

• Kinect Drag and Drop
• NUI
• Flying Text
• Multidimensional

Project Information URL: http://tomek.kownet.info/projects/humanbody.html

Project Download URL: http://sdrv.ms/Ji2vEZ

Contact Information:

• Blog: http://tomek.kownet.info/
• Twitter: @tomkowalczyk

• [0:27] Windows 8 Release Preview
• [1:28] Visual Studio 2012 Release Candidate is here
• [3:04] Welcome to the Developer Tools Blog Network (Lisa Feigenbaum)
• [4:59] My Favorite Features: Creating Storyboards with PowerPoint (Jason Zander)
• [6:29] Being productive in the background – background tasks (Hari Pulapaka)
• [8:29] Getting Started on Line-of-Business HTML5 App – Part 4 Using Modernizr, Polyfills, YepNope (Bruce Kyle)
• [10:41] Howard Dierking Shows You How to Optimize Your ASP.NET Sites (Brady Gaster, Howard Dierking)

Picks of the Week!

• Rick's Pick of the Week: [11:20] Fun with HTML5 Canvas, WebSocket, JQuery and ASP.NET. End-result: A live white board on a web page! (Al-Farooque Shubho)
• Clint's Pick of the Week: [12:47] Occlusion detection and virtual object manipulation with the Kinect

Occlusion detection and virtual object manipulation in augmented reality with the Microsoft Kinect

Augmented reality (AR) is limited by how you can interact with the virtual object displayed. Currently, the object can only be moved by moving the AR marker. In this project, we used the Microsoft Kinect sensor to allow one to virtually touch and manipulate the object displayed. In order to do this, we use the skeletal tracking of the sensor to allow the user to rotate and resize the object with his hands. While doing so, the user hands will inevitably get between the virtual object and the camera. To prevent the virtual object from being displayed wrongly over the hand, the depth map of the Kinect is used to find any obstruction and block rendering of pixels under occlusion. This is realized at the GPU level, using modified Shaders. The AR system uses the Unity3D game engine to display the 3D models with a custom plugin created at the CIMMI research center to enable the AR. The plugin computes in real time the pose of the Kinect using the color image while the depth map allow occlusion detection and hands tracking. The plugin was developed with the OpenCV library which allows easy analysis of the images. This work was realized at the CIMMI research center (cimmi.qc.ca) by two interns, Alexis Legare-Julien and Renaud Simard, under the supervision of Jean-Nicolas Ouellet, Ph.D.Eng.

 

Here are some additional details on the project:

It uses the Kinect color image and depth map to achieve augmented reality with occlusion and object manipulation. Without wasting time, here is the link to the demo (April 2012): http://youtu.be/OTmHtAaQD_c 

Now the description:

We work on augmented reality (AR) where a virtual object is added to a live video stream as if it was really part of the scene. Last year, we developed unity3d plugin that is able to create the AR illusion. Basically it grabs the webcam image and computes the camera position wrt. a marker. Unity then only shows the webcam image superimposed with a render of the virtual object from the correct view point. This video shows what it looks like: http://youtu.be/npuubLGpU1A 

Then we wanted to allow the user to manipulate the state of the object, rotate and resize, with his hands. The Kinect is perfect for that, the skeletal tracking measure where the user is relative to the camera (hand, head, etc). While doing so, we were confronted with the occlusion problem. That is, if the hand is closer to the camera than virtual object, the hand must be seen (we must not render the virtual object over the hand). Again, the Kinect measure the distance of everything it sees, the depth map, the z-distance. Similarly, when a video card renders a virtual scene it computes occlusion from the depth buffer, i.e. the z distance of every object in the virtual scene. The occlusion algorithm is then obvious, on the shader clip( z.real < z.virtual). 

Thus, we used the Kinect (with the Microsoft sdk, but any other sdk with skeletal tracking would do) to manage occlusion and manipulate the virtual objects interactively. Rendering is done in Unity3d free, occlusion is managed in a unity shader (this is very fast since computation is done on the graphic card), manipulation is done in Unity3d from skeletal information from the Kinect. 

The biggest problem we faced was speed, transferring the depth map to the shader was time consuming. We had to resize the depth map to 512x512 to allow faster manipulation. While doing so, we were confronted by a new problem; the occlusion clip was highlighting the object contours. So occlusion was correct but when the hand was behind the virtual object we could see through the virtual object at the object boundary. The effect was kind of cool but unwanted. 

After some investigation we found out that opencv was using bilinear interpolation to resize the depth map image. This was creating intermediate values between the background and the object boundaries. Forcing to nearest neighbor interpolation was the solution, but the contours were still there. The shader was interpolating the values when we extracted the depth value from the texture. The coordinates are indexed from 0 to 1 instead of 0 to width (or 0 to height). We had to crop the decimal value of the coordinate to make wure it was falling at the center of the texture pixel to avoid interpolation. While doing so we found out that unity3d is indexing pixels coordinates from the pixel corner, not the center. An offset of 0.5 must be added to the coordinate in the native image range. 

Project Information URL: http://www.youtube.com/watch?v=OTmHtAaQD_c&feature=youtu.be

Monster Mixer - Medialogy Aalborg University P2 2012 - Kinect Unity game for dyslexia

Monster Mixer is a game made by students from the 2nd semester of Medialogy, Aalborg University.

TARGET GROUP: Danish kids age 6-8

PROBLEM STATEMENT: How can Kinect be used in order to create a game that reduces the impact of dyslexia by training phonological awareness?

Monster Mixer is a Kinect game developed using Unity game engine. Its purpose is to raise children's phonological awareness and thereby preventing dyslexia that they might develop in the future.

The game uses monsters and nonsense words to train phonological awareness. It requires gestures to catch monsters. The goal is to combine the right monsters into a new monster/sound. Nonsense words are used to make it impossible for the kids to just recognize already-known words.

...

We used the Zigfu Unity wrapper to make Kinect work with the Unity engine.

Project Information URL: http://www.3dbuzz.com/vbforum/showthread.php?192900-Monster-Mixer-a-Unity-Kinect-game-my-university-group-have-made

(I'm not sure what this says about me, but when I saw this, my first thought was, "Hey, that's like the chain gun in the Apache helicopter!")

Servo Face Tracking with Kinect SDK v1.5

This is a quick-and-dirty experiment using the Kinect SDK v1.5 (released on Monday 5/21) with face tracking features. The laptop I'm using here is incredibly under powered for the job so the results are poor, but I wanted to capture what I was able to cobble together today and share it. The webcam in the video is mounted to two servos that are powered by some electronics I breadboarded for an earlier experiment. I still have some work to do on the servo responsiveness, but I like where this is going. The baseline project is part of the new SDK Developer Toolbox (Face Tracking Basics in C#).

Project Information URL: http://www.youtube.com/watch?feature=player_detailpage&v=EPWxGrKn-r0

Contact Information:

• Blog: http://bkbrown.com/index.html

Kinect Angles v2.4

This release features a new camera sound when the game takes a picture, improved keyboard entry for high score names and other minor bug fixes. It is for Kinect SDK v1.0.3.190 or the new v1.5. I have decided to stop updating the beta version and concentrate on releases which work with the new commercial SDKs.

Project Information URL: http://drenton72.wordpress.com/2012/05/23/kinect-angles-v2-4/

Project Download URL: http://bit.ly/JpTGrW

Project Source URL: http://bit.ly/KTqDx8

Contact Information:

• Blog: http://drenton72.wordpress.com
• Twitter: @drenton72

Kinect SDK 1 - Depth and Video Space

In previous chapters we learned how to read in video and depth data. Now we take a break from looking at how to extract data from the cameras and concentrate on how to relate data from the depth camera to the video camera.

The Kinect has two cameras, one video and one depth. In both cases they return a bitmap corresponding to what they see. In the case of the video camera the bitmap is usually 640x480 and for the depth map it is often 320x240, i.e. exactly half the resolution.

A very standard requirement is to pick out video pixels that correspond to particular depth pixels. This sounds fairly easy as you might think that the pixel at x,y in the depth image corresponds to the four pixels at 2x,2y in the video image.

Unfortunately this simple mapping doesn't work and this can be the cause of much wasted time in trying figure out why your perfectly reasonable program doesn't quite do what you expect. Fortunately, the solution is fairly easy - once you know how.

As a simple example of using the data from the depth camera with the video camera we construct a demo of how to use the player index (see chapter 4) to create masks that can be used to remove the background. from a user's image.

...

Converting from depth to video

To convert from depth to video coordinates is simply a matter in projective geometry. What we have are two perspective views of the same scene and so it is perfect possible to implement a function which converts between them - possible but not easy to get right.

For this reason the DepthImageFrame object has a set of coordinate conversion methods. In this case we need the MapToColorImagePoint method. This takes the depth image coordinates and takes into account the depth at the point and converts to the location in the video image. All you have to tell the method, in addition to the depth co-ordinates is the format of the video image.

(Note: you don't have to supply the depth of the pixel to the methods as in the early beta SDK.)

Next we need to compute the video co-ordinates vx,vy from the depth co-ordinates x,y:

...

Project Information URL: http://www.i-programmer.info/ebooks/practical-windows-kinect-in-c/3912-kinect-sdk-1-depth-and-video-space.html

Project Source URL: http://www.i-programmer.info/codebin.html (Registration required)

void FramesReady(object sender, AllFramesReadyEventArgs e)
{

DepthImageFrame DFrame = e.OpenDepthImageFrame();


if (DFrame == null) return;


short[] depthimage = new short[DFrame.PixelDataLength];


DFrame.CopyPixelDataTo(depthimage);

ColorImageFrame VFrame = e.OpenColorImageFrame();


if (VFrame == null) return;


byte[] pixeldata = new byte[VFrame.PixelDataLength];


VFrame.CopyPixelDataTo(pixeldata);

byte player;
int vx, vy;

for (int y = 0; y < DFrame.Height; y++)
{
for (int x = 0; x < DFrame.Width; x++)
{
player = (byte)(depthimage[
x + y * DFrame.Width] & DepthImageFrame.PlayerIndexBitmask);
if (player != 0) player = 0xFF;

vx = x * 2;
vy = y * 2;
ColorImagePoint p = DFrame.MapToColorImagePoint(
x, y, 
ColorImageFormat.RgbResolution640x480Fps30);
vx = p.X;
vy = p.Y;

vx = Math.Max(0, Math.Min(vx, VFrame.Width - 2));
vy = Math.Max(0, Math.Min(vy, VFrame.Height - 2));

for (int k = 0; k < 8; k++)
{
pixeldata[(vx + vy * VFrame.Width) * VFrame.BytesPerPixel + k] &= player;

pixeldata[(vx + (vy + 1) * VFrame.Width) * VFrame.BytesPerPixel + k] &= player;
}
}
}
pictureBox1.Image = ByteToBitmap(pixeldata, VFrame.Width, VFrame.Height);
}

Contact Information:

• Blog: http://www.i-programmer.info/
• Twitter: @Iprogrammerinfo