Holographic Window using Kinect V2
I found today's project via Clemente Giorio's tweet and once I watched the video, there was no question about me highlighting it here as soon as possible... While Hololens might make this look like child's play, we don't have them yet and we DO have Kinect's... :)
Real life portals without trickery
The Kinect V2 is a sensor that can be used to record a 3D view of the world in real-time. It can also track users and can see what their body pose is. This can be used to perform head tracking and reconstruct a camera view into a 3D world as if looking through a virtual window. By using one Kinect for head tracking and another Kinect for reconstructing a 3D view, the virtual window effect of a portal can be created in reality. By using both Kinects for 3D world reconstruction and head tracking a two way portal effect can be achieved.
In the setup used for recording the video two Kinect V2 where used. My laptop was connected to a projector that projected onto the wall. The PC displayed the other much smaller portal. Smaller portals hide more of the other side of the scene and allow for larger head movements. With a bigger portal you will run into the limitations in the field of view of the Kinect much earlier.
Instead of transferring the recorded 3D scene I swapped the Kinects and only transferred the recorded bodyframes through a network connection to mimimize latency. This limits the maximum range that the portals can be placed from each other. (about 7 meters when using USB3 extensions cables)
A portal opens as soon as a user is detected by the Kinect so proper head tracking can be done. For the video I used the right hand joint for controlling the camera so the viewer would experience what it would look like when head tracking is applied.
Holography is quickly becoming the buzzword of 2015. It’s getting harder to keep a clear understanding of what holographic actually means. (and yes I’ve abused the term too) I like the view that Oliver Kreylos has on the term holography. (See: What is holographic, and what isn’t?)
Since both worlds are already rendered in 3D it is a small step to add stereo rendering. For instance with a Microsoft HoloLens. This brings us closer to a holographic window.
Here’s the checklist that Oliver Kreylos uses in his article:...
Project Information URL: http://smeenk.com/real-life-portal/