Kinecting with F# (Channel 9)

Kinect for Windows SDK F# examples are pretty rare (we've only had one other so far, Kinect SDK and F#) so I wanted to be sure to get this one into the Gallery. It's short and sweet, but does a good job of showing off the power of F# when used with the Kinect.

Kinect SDK with F#

I finally got around to taking a look at the Kinect SDK the other day, partly because I was interested to see how the API looked from F#. ...
Eventually, I got Win7 x86, Visual Studio and the Kinect SDK installed on the metal, plugged in the sensor and – whoa – the devices were recognized and the drivers installed and… the samples ran!
From this point doing some hacking was pretty straightforward. I set about creating a project that used the skeletal tracking ability from the SDK. The project needs to reference the Microsoft.Research.Kinect assembly, then we can open some namespaces and initialise the API:
...
The API uses .NET events to communicate back to the application that some form of data is available. Depending on the options that you specify at init time, any of the skeletal, depth frame or colour data will be returned in the event arguments. This seems like a good place to use F# Async’s event integration: Async.AwaitEvent. We can quite easily write some code that will create an Async task that will repeatedly listen for events:
..."

Project Information URL: http://www.voyce.com/index.php/2011/09/05/kinect-sdk-with-f/

open System
open System.Windows
open Microsoft.Research.Kinect

[<STAThread>]
do
    let nui = Nui.Runtime()
    nui.Initialize(Nui.RuntimeOptions.UseSkeletalTracking)

    // lifted straight from the sample code
    let getDisplayPosition w h (joint : Nui.Joint) =
            let mutable depthX = 0.0f
            let mutable depthY = 0.0f
            nui.SkeletonEngine.SkeletonToDepthImage(joint.Position, &depthX, &depthY)
            let depthX = depthX * 320.0f; //convert to 320, 240 space
            let depthY = depthY * 240.0f; //convert to 320, 240 space
            let mutable colorX = 0
            let mutable colorY = 0
            let iv = new Nui.ImageViewArea()
            // only ImageResolution.Resolution640x480 is supported at this point
            nui.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(Nui.ImageResolution.Resolution640x480, iv, 
                (int)depthX, (int)depthY, 0s, &colorX, &colorY)

            // map back to visible area
            new Point((w * (float)colorX / 640.), (h * (float)colorY / 480.))

    // Set-up the WPF window and its contents
    let width = 1024.
    let height = 768.
    let w = Window(Width=width, Height=height)
    let g = Controls.Grid()
    let c = Controls.Canvas()
    let hd = Shapes.Rectangle(Fill=Media.Brushes.Red, Width=10., Height=10.)
    ignore <| c.Children.Add hd
    ignore <| g.Children.Add c
    w.Content <- g

    // We simple move the rectangle to where the head is
    let draw (s : Nui.SkeletonData) =
        let point = getDisplayPosition width height s.Joints.[Nui.JointID.Head]
        ignore <| w.Dispatcher.BeginInvoke(Threading.DispatcherPriority.Normal, Action(fun () ->    
            hd.SetValue(Controls.Canvas.TopProperty, point.Y)
            hd.SetValue(Controls.Canvas.LeftProperty, point.X)))

    let skeleton (nui : Nui.Runtime) =
        let rec loop () =
            async { 
                let! args = Async.AwaitEvent nui.SkeletonFrameReady
                args.SkeletonFrame.Skeletons 
                |> Seq.filter (fun s -> s.TrackingState = Nui.SkeletonTrackingState.Tracked)
                |> Seq.iter draw
                return! loop () 
            }
        loop ()

    skeleton nui |> Async.Start

    let a = System.Windows.Application()
    ignore <| a.Run(w)

    nui.Uninitialize()

Contact Information: