Home Automation, Kinect powered


Today's project is from Friend of the Gallery, Dan Thyer and how he's improving his home automation system with some help from the Kinect, the Kinect Point Cloud, Netduino and more.

Home Automation with Microsoft Kinect Point Cloud and Speech Recognition

I love using the Microsoft Kinect for my home automation projects. The Microsoft Kinect for Windows API is really amazing and can be used to track our movement in the physical world in unique and creative ways outside of the traditional game controller. Traditional computer vision systems are too slow to track normal human motion, but the Kinect is able to give you coordinates of 20 joints 30 times a second. The Kinect is able to simplify the computer vision problem by creating what is called a Point Cloud out of infrared light. This infrared light is similar to visible light but has a longer wavelength than what we can see. The Point Cloud is able to be seen with a special camera or night vision goggles as shown in the image below.


The Kinect has a special lens that sends out a known pattern of spaced lines of infrared light. The light makes dots on the objects it touches creating a Point Cloud. The Kinect has a special camera for seeing the infrared dots. The vision system on the Kinect measures the distance between the dots and analyses the displacement in the pattern to know how far away an object is. See the image below to see how close up objects have dots closer together and further objects have dots spaced further apart. The Kinect is able to analyze the spacing of the infrared dots to build a depth map and quickly see a human outline because the human is in front of other objects.


Create a Natural UI with the Kinect

There are some great user interfaces with the Kinect but most require you to be looking at a computer screen. I built a system that does not require you to look at a computer in order to select a device and turn it on or off. You can simply point to a device with one hand and raise your other hand above your head and wave one direction to turn on and the other direction to turn off. In addition to using gestures, I use the Kinect speech recognition engine to turn devices on or off.



Z-Wave is a wireless communications protocol designed for home automation. There are tons of commercially available Z-Wave devices. The LogicalLiving.ZWave project contains a Class Library to control Z-Wave devices through the Areon Z-Stick Z-Wave USB Adapter. I purchased the USB adapter online for less than fifty dollars and all of my Z-Wave devices were also each under fifty dollars. I installed the Z-Wave devices by turning off the power in the house and then replaced the standard light switches and power outlets with Z-Wave devices. I wrote the LogicalLiving.Zwave.DesktopMessenger project to be a sample Windows Forms UI to control the LogicalLiving.ZWave Class Library. It is useful to use the LogicalLiving.Zwave.DesktopMessenger to figure out the values for the Z-Wave DeviceNode. Each Z-Wave device has its own unique DeviceNode which is required in the Z-Wave message to change its state.


Netduino is a wonderful open-source electronics prototyping platform based on the .NET Micro Framework. I use the netduino plus 2 and a custom circuit that I built to control many devices in my home including turning on my fireplace, aiming a squirt gun at the pool, watering the garden and opening the garage door. I use the Kinect.Living gestures and audio commands for turning on the fireplace. We have a new kitten in the house who is very interested in the fireplace. I quickly became concerned that the kitten would crawl into the fireplace at the wrong time while someone was doing the gesture or audio command to turn it on. For safety, I wired up a mesh screen curtain that she cannot get behind! Please read my previous articles on the netduino and jQuery Mobile:


It is really fun to use the Microsoft Kinect for Windows API for home automation projects. This project presents a much more natural UI for controlling your devices in your house. It is really nice to be able to control devices without needing a remote control. In our house, the remote control is always lost in the sofa somewhere, but no worries anymore. With the Microsoft Kinect and this project, you are the remote control to control the devices in the entire house.

The ideas in this article can reach far beyond home automation. There are many other useful applications for having a computer know what object you are pointing to. We are living in an exciting time where vision systems are packaged up into inexpensive devices and are readily available such as the Microsoft Kinect. The Kinect and the Kinect for Windows SDK enables us to build incredible applications with minimal effort that would seem like science fiction 10 years ago.

[Click through to the full post to see all the details, source and more]

Project Information URL: http://www.codeproject.com/Articles/715858/Home-Automation-with-Microsoft-Kinect-Point-Cloud

Project Download URL: http://www.codeproject.com/Articles/715858/Home-Automation-with-Microsoft-Kinect-Point-Cloud

Project Source URL: http://www.codeproject.com/Articles/715858/Home-Automation-with-Microsoft-Kinect-Point-Cloud

Contact Information:

The Discussion

  • User profile image

    Thanks Greg for the post about my project!  

    I just extended the home automation project with an Internet of Things cat toy with a servo controlled laser and a servo controlled mouse puppet controlled by netduino plus 2, the Kinect, and has a mobile app too.  I'll post a video online soon and will be demoing the project at Carolina Code camp in Charlotte NC, plus at CodeStock in Knoxville TN and at an Internet of Things Lunch and Learn at Logical Advantage in Charlotte NC.  Anybody interested can tweet me at @logicaldan for the details. 

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.