Kinect4NES (Yes, Kinect to a Classic NES)
Paul DeCarlo recently posted a project that seems so outside-the-box that I couldn't help but highlight! :)
Recently, I have found myself becoming involved in the exciting world of “IoT” or internet of things. All of this started while attending a presentation on the subject that was put on by my fellow colleague Bret Stateham. If you are unfamiliar with the concept of “IoT”, I like to think of it as a programmable system comprised of input sensor(s) / polling service(s) which interact with a physical device and optionally store data received by the sensors to a web service where it could be optionally processed for patterns to facilitate things like forecasting. In short, we are connecting your things that are inherently offline to the internet.
This particular project does not quite fully fit the definition above as the end result is a non-network connected thing (a classic NES console) connected to modern sensor (the Kinect V2). However, it could be easily modified to fit such a description. For example, this concept could be extended to allow the public to access and control a physical NES through a web interface (think Twitch plays Pokémon) ~Coming Soon~ OR it could allow users to upload Kinect Gesture profiles online that could be pulled down through the application to allow better control in certain games. Nonetheless, it leverages concepts that are integral to most “IoT” projects, specifically hardware interface construction, software interfacing, and application development.
I’d like to elaborate a little bit more on inspiration for this project as I would really like to tear down any barriers currently holding back abled developers from breaking into this field. At the end of Bret’s presentation, he showed off a quick demo that showcased an Intel Galileo board running Windows for IoT that controlled a blinking LED with breakpoints set in Visual Studio. Not exactly, jaw dropping surface value, but when looked at for what it can enable rather what it is specifically doing, you may find an opportunity to expand the possibilities of your code. It dawned on me that that this blinking LED demo was all I needed to know to allow computer code to interact with physical objects. I began thinking about everything like a blinking LED project, SMS notifications from my washer/dryer/dishwasher when a cycle is complete, automating the addition of chemicals to a swimming pool, or firing a rocket when a threshold of retweets is achieved on a particular hashtag. All of these become comparatively simple problems when looked at through the lens of turning on a light when a certain condition is met! I soon found myself pondering the idea of mixing old nostalgic technology with the bleeding edge. What if I could control a classic NES with a Kinect 2 device? Not through an emulator, but a physical, rectangular, Gray-Box NES from 1984.
- An NES console with game to test
- An NES controller OR some wiring skills and a CD4021BE 8-bit shift register
- 12 strands of wire, recommend Kynar
- 8 1k resistors (technically any value from 1k to 50k should suffice)
- 2 3.6k resistors (again higher not necessarily bad)
- IoT board capable of running Firmata, Intel Galileo or Arudio Uno etc.
- Kinect V2 Sensor
- Machine capable of running the Kinect V2 SDK
In my write-up, I am going to assume you have zero experience with hardware development which is fitting because I literally had no idea how to even blink an LED when I started this project last week. I am also going to assume you want to know how to do achieve the final product from a blank slate, let’s start by breaking down the problem into sub-problems.
We want to use a Kinect V2 Sensor to control games on a physical NES
1. We need to interface with the NES controller port using computer code
2. We need to speak to that hardware interface through a software interface, preferably in C#
3. We need to create an application that takes input from the Kinect V2 Sensor and processes it through the software interface, into the hardware interface, where it can reproduce button presses on the NES console based on defined gestures.
Once you have all this, put all the pieces together and turn on your favorite game! I chose the pinnacle classic Super Mario 3 which worked well enough with our scheme to actually allow you to play through the first level! Next thing to consider is trying other games out and possibly allowing for multiple gesture profiles. For example, I have created a stub in the hosted project to play Mario by physically jumping and running as opposed to using hands. All in all, this was an extremely fun hack that allowed me to bridge my interest in class video games with modern gaming peripherals!
If you want to get the bits and follow along with updates or even contribute to this project, you may want to check out the GitHub Project Page for Kinect4NES.
Project Information URL: http://pjdecarlo.com/2014/10/kinect4nes-control-your-classic-nes-with-the-power-of-kinect-v2.html
Project Source URL: https://github.com/toolboc/Kinect4NES