Steven Edouard has revisited his KinectKannon, highlighted here, Kinect Kannon: Kinect-Augmented T-Shirt Cannon, with a much long and indepth blog post (and source!)
Today a I finally had some free time to write up a detailed overview on the KinectKannon. Enjoy!
The Kinect Kannon has been put together with mostly store-bought parts. The parts list includes:
- AirCannons.com T-Shirt Cannon
- Phidgets USB Interface Relay Kit
- Phigets USB Advanced Servo Controller
- ServoCity.com MPT2100 Pan and Tilt Module
- Hard Oak Plywood Box Hand Assembled
- Kinect For Windows and Kinect for Windows SDK
The TShirt Cannon is mounted to the Pan and Tilt control with a piece of 2 by 4 wood. This keeps the cannon attached firmly allowing for more precise control.
To secure the Pan/Tilt control, a wooden box was made from oak plywood. This provides a very sturdy base for the control as it will be shifting its center of gravity as the cannon is moved and aimed.
An empty space is provided in the box with a missing wall. Inside are a few things:
A WPF (Windows Presentation Foundation) desktop application is used with the Kinect SDK as well as some auxiliary libraries for the XBOX controller as well as the servo and relay controllers. The application is organized using a high level code behind file for the Main XAML window and a few singleton classes:
- PanTiltController – A singleton class which provides high level on/off and panning methods.
- FiringController – A singleton class which provides a ‘virtual’ safety as well as a high level firing method.
- Automator – A small class that handles controlling the PanTiltController based on audible or skeletal input.
- Renderer – A class that handles rendering the color, infrared and skeletal streams.
- HudRenderer – A class that handles rendering the Heads-up Display
- App – The main app file which controls high-level states of the application such as tracking mode and the state of the safety.
Like any other engineering project, we were under a tight timeline. We had approximately 4 weeks before we needed to showcase it at a large presentation. We chose the path of least resistance.
The Kinect SDK works only on Windows, although there is an open source implementation that can be used on Mac OS X and Linux. The SDK works out of the box seamlessly with the Kinect and already has very nice wrappers for C#. We decided to use the SDK as it provided samples in C# for skeletal, audio and color frame rendering. It made things way faster by simply taking the code examples and organizing them into a larger application.
Why WPF and not a modern Windows 8 App? .NET and Windows APIs have a different surface area for the Windows 8 app environment. In this environment not all APIs available on the desktop are accessible. Although the Kinect SDK v2.0 supports Windows 8 apps, the drivers for the servo controller, relay controller and possibly the open source library for the XBOX controller had dependencies on legacy desktop APIs. We also have no intention of publishing the app on the store because the Kinect Kannon is quite expensive!
The Kinect Kannon is easy to operate but requires a few key pieces of knowledge and warnings.
Ensure that you have a fast dedicated machine to use. You need an i7 Surface Pro 3, Lenovo X1 Carbon or faster.]
Install the following:
- [Kinect for Windows SDK]https://www.microsoft.com/en-us/download/details.aspx?id=44561)
- Phidgets USB .NET Libraries and Drivers
- Visual Studio/VS Express 2013 (or newer)
When you start up the Kinect Kannon you will be presented with the Camera (Color Stream) and Heads Up Display (HUD).
Kinect Kannon has 3 operational modes:
- Manual Mode
- Skeletal Tracking Mode
- Audible Trackign Mode
In manual mode you can control Kinect Kannon on the X and Y axis by moving the left analog stick. This mode is the default mode of Kinect Kannon and can be reached by pressing X on the XBOX control.
In skeletal mode, you cannon control the Kinect Kannon in either the X or Y direction. The Kannon will move to track any detected skeletons in its view. To change the targeted skeleton press Left and Right on the directional pad (this only works if there is > 1 skeleton on the screen). Press A on the XBOX control to get to this mode.
Audible mode prevents you from controlling Kinect Kannon in the X direction but you can control it in the Y direction. Kinect Kannon will autonomously pan left and right depending on point sounds it recognizes from its directional microphone. Press B on the XBOX control to get to this mode.
The Kinect is equipped with an infrared sensor and therefore can see at night. Press Y on the controller to activate night vision mode. This doesn’t change any functionality of Kinect Kannon – skeletal and audible tracking work the same.
The video window will be a bit smaller because Kinect’s infrared resolution is smaller than its camera.
Firing the Kinect Kannon
Project Information URL: http://stevenedouard.com/inside-kinectkannon-semi-autonomous-robotic-t-shirt-cannon/
Project Source URL: https://github.com/sedouard/KinectKannon