Kinect to HoloLens Holograms
Joost van Schaik shares a great project that meshes the old and new worlds (so to speak).
There are several demos out there that show obviously scanned models of people or physical objects used in HoloLens applications. I think it was fellow MVP and Dresdener 3D genius René Schulte who first used his bust in a demo or at least went public with it. Unfortunately I have not been able to find very much about the actual process used to do the scanning and get to this result, so I have been trying to kludge together a procedure to make a full color 3d scan of a physical object – myself – and show it in a HoloLens.
For this you will need the following hardware:
- A PC with a USB3 port. A Surface 4 Pro i7 has been successfully used for this goal
- A Kinect sensor for Xbox one, aka Kinect 2
- A Kinect Adapter for Xbox One S and Windows PC
And the following software:
- The 3D Scan app (see below)
- CloudCompare Stereo
- This special shader that I nicked from this thread on the Unity forms
Setting up the hardware...
Downloading additional software...
The actual scanning
Converting to a format usable for Unity
Importing in Unity and showing the colors
- First of all – already mentioned in passing, make sure there is enough light, and prevent shadows. This is harder than it sounds. You will notice scanned Holograms tend to look rather pale when created with insufficient light.
- The handheld setting is way harder to use than the stationary setting. Consider placing objects on a rotary platform rather than moving around it with the Kinect
- Move or rotate the object you want to scan slowly (or move the Kinect slowly)
- You will have to fiddle a lot with settings before you get the result you want
- Larger objects (like humans) are way easier to scan than small objects
- Make anything you don’t want on the scan as black as and non-reflective as possible
Unity will complain regularly about an error in the shader in the editor, but it still seems to work fine. Having no clue about shaders and how to write them yet, I tend to ignore it. The resulting project, although containing no code written by me at all, can be found here.
Project Information URL: http://dotnetbyexample.blogspot.com/2017/01/scanning-physical-objects-with-xbox-one.html
Project Source URL: https://github.com/LocalJoost/KinectScanDemo