"Kinect Client Server System"
- Posted: Nov 29, 2012 at 6:00AM
Today's project shows off one means of connecting a WindowsRT app to a Kinect for Windows device (And shows off a bunch of other cool things like some DirectX goodness and more)
The Kinect Client Server system consists of a desktop application and a WinRT application. The desktop is a server that sends Kinect for Windows data over a WinSocket2 socket. The WinRT application is a WinRT client that connects to the server to receive and show the Kinect data.
You can download the server binary from here, and run it. You can also download a compiled version of the client (named 3D-TV) from here directly. You will need a developer license to install it on your Windows 8 pc. Only the x86 platform is currently supported. An entry in the Windows store for the client will come in time (removes the requirement of a developer license).
You can also download the source code and build the system yourself. The client application requires the WinRT DirectX Bus to build. So you will have to download that one too and use it as described. The Project file assumes that the TheByteKitchenLibs directory (either the source code or the compiled version) is a sibling of the KinectClientServer directory.
Project Download URL: http://thebytekitchen.codeplex.com/releases
Project Source URL: http://thebytekitchen.codeplex.com/SourceControl/changeset/view/20507
The Kinect SDK is not compatible with WinRT in the sense that software developed using the SDK cannot have a WinRT (Windows RunTime) UI. The reason is that the Kinect SDK is .Net software and you cannot run (full) managed code on the WinRT.
Nevertheless, I want to create software that can show Kinect data in a WinRT UI. For multiple reasons, one being that software written for the WinRT can run on a PC, a tablet, very large screens, now called a Surface, and a Windows Phone. A survey of other solutions, see below, reveals that solutions to this problem are based on networking. Networking allows us to deliver Kinect data anywhere. This then is another reason to work on separating the source of Kinect data from its presentation.
The general solution is to make a client-server system. The server lives in the classic Windows environment, the client is a WinRT app. Communication between client and server is realized using networking technology; preferably the fastest available. The server receives the data from the Kinect and does any processing that involves the Kinect SDK. The client prepares the data for presentation on the screen. If multiple servers are involved, it integrates and time-synchronizes data from several servers. Since I’m a C++, DirectX guy, the server and client are built on just these platforms
In theory, WebSockets are slower than WinSockets. There can be much discussion about what would be the fastest solution under which circumstances. I expect WinSockets to be the fastest solution, therefore I prefer WinSockets.
Also, in theory, a C++ program is faster, and smaller, than an equivalent C# program. There can be much discussion … , therefore I prefer a program written in C++.
Of course, we should do asynchronously, or in parallel, whatever can be done quicker in parallel.
So, what’s a smart way to develop a client server system to show Kinect color and depth data in a WinRT app? For one, we start from SDK samples:
- A sample from the Kinect SDK that elaborates processing depth and color data together.
- A sample the shows how to use WinSockets (in C++).
- A sample that show how to use the WinRT StreamSocket using PPL tasks (yes we will exploit parallelism extensively .
- Windows Service (optional, see below).
The use of a Windows service is an option for later use. To work with a service instead of a simple console application requires that the server is capable of handling all kinds of exceptional situations, if only by resetting itself. Consider e.g. the case that no Kinect is connected, or if the Kinect is malfunctioning? Etc.? And apart from that, I expect the Kinect SDK to be made available for WinRT applications in due time.
Project Information URL: http://thebytekitchen.com/2012/10/17/viewing-kinect-data-in-the-new-windows-8-ui/