That's Ripplelicious

I'm not sure how to describe this, it's pretty darn cool visual effect. I'll let the video speak for itself. This would be fun at a party

Kinect Ripplelicious

The way the application works is that it displays whatever the RGB camera sees, and tracks the skeletons of up to two people in the frame, and checks which of their joints are closer than a certain distance to the Kinect sensor. Those joints should cause a ripple in the video image, so the joints 3D coordinates are converted to 2D pixel coordinates corresponding to the joint's position in the video image.

At these 2D coordinates, a ripple pixel shader effect is displayed. Pixel shaders are small C procedures that run on the GPU, and I have no idea how they work exactly, so I downloaded Shazzam and let it generate a nice C# wrapper class for me. Shazza also allows you to test the shaders, so I played around with the provided Ripple pixel shader's properties. I decided that each ripple effect would start with a certain frequency (the number of ripples in the effect) that would over time decrease to a frequency of zero meaning no ripples. This would give the impression of the ripples calming down again.

Each ripple effect also has an amplitude, which determines how 'deep' the ripples are. A light touch should cause a light ripple and a deep punch should cause a wild, deep ripple, so the closer the joint is to the sensor, the higher the ripple's amplitude.

...

Project Information URL: http://channel9.msdn.com/Forums/Coffeehouse/I-love-you-Kinect#c1eb66d8f52f541559e899f0b0068ab1b

Project Download URL: https://skydrive.live.com/redir.aspx?cid=62971dd1548e5f8d&page=play&resid=62971DD1548E5F8D!212

void nuiRuntime_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
    // Get all tracked skeletons
    var trackedSkeletons = (from s in e.SkeletonFrame.Skeletons
                            where s.TrackingState == SkeletonTrackingState.Tracked
                            select s);

    // Iterate through all tracked skeletons and grab the joints that are close enough to cause ripples.
    List<Joint> joints = new List<Joint>();
    foreach (var skeleton in trackedSkeletons)
    {
        
        joints.AddRange(from Joint j in skeleton.Joints
                        where j.TrackingState == JointTrackingState.Tracked && 
                        j.Position.Z <= maxZDistance
                        select j);
    }

    // For each joint that causes a ripple...
    foreach (var joint in joints)
    {
        // Convert the joint's 3D skeleton-space coordinates to pixel coordinates, so that we know where the joint is in the video image.
        Point ripplePosition = nuiRuntime.GetDisplayCoordinates(joint, videoImage.Width, videoImage.Height);

        // Count the number of ripples currently animating that are too close to the current ripple position.
        var rippleCount = (from r in ripples
                           where DistanceBetweenPoints(r.Position, ripplePosition) < minRippleDistance
                           select r).Count();

        // If no ripples are too close, we can safely add a new ripple.
        if (rippleCount == 0)
        {
            // Calculate the ripple's amplitude based on how deep the joint is pushing (in other words, how close it is to the sensor.
            double amplitude = Math.Abs((joint.Position.Z - maxZDistance) / amplitudeCoefficient) + minRippleAmplitude;
            Debug.Assert(amplitude <= maxRippleAmplitude);

            AddRipple(ripplePosition, amplitude);
        }
    }
}

Contact Information:

• http://channel9.msdn.com/Niners/Bas