Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Recreating the Kinect Hub as a means to learn how to code with the Kinect

Today's project provides another learning experience, one where learning how to emulate/recreate the Kinect Hub on the XBox 360 is used to learn how to code with the Kinect and Kinect for Windows SDK

Xbox Kinect Hub Remake - An Introduction To New User Interfaces

I first started playing around with my Kinect at home just to get a feel for what was involved in working with the SDK – I must say that I, like many of you, was amazed, and still am, at how awesomely simple and beautifully designed the API’s in the Kinect NUI framework are. A lot of people have written little Kinect demo’s showing how to create buttons and detect hand location, but I thought I’d try something different. So I set out to mimic some of the Xbox Live's interface elements in WPF.

I must admit this post might be a little late in it’s timing – most people where blogging about the Kinect SDK at the very start of this year, and while I was part of that throng of developers excitedly crawling all over the SDK like a pack of red-cordial infected pre-school children who’ve just discovered a secret stash of unattended Space Hoppers, I simply haven't had the time I'd like to write about it. I cannot contain that silence any longer.

Game Plan

Before you set off on any little learning adventure you usually want to have a set goal in mind to make your learning experience more driven – for me this was accomplishing something different to a lot of the tutorials out there and that was to create a unified interface that closely matched the Xbox Kinect Hub that you use to navigate around your Xbox with.

...

But basically, I want something similar to the Kinect Hub (screen shots below).

Primarily I want to create an interface that has a tile based layout and allows you to “press” the buttons with your hand. This would then allow me to create applications that allow you to “navigate” around a series of pages and view “assets” – whether they be videos or images. The navigation sections won’t be covered in this post, but I plan on doing a number of other posts covering these functionalities.

I also want a “hand” cursor that has a timer based “ring” that is used to “press” the buttons similar to the screen below:

...

We are ready to get going, and the first thing i want to do is implement the Xbox Kinect Hub “hand”.

This is an interesting part of the project, as to solve it I’m going to hack a few different things together. The hand needs to have a circle complete a rotation around it when it hovers over an object. Clint Rutkas has already completed similar functionality in the Coding4Fun Kinect project with the addition of Hover Buttons that have a relatively standard eventing model, although he doesn’t use a hand as his button, and instead uses simply circular buttons. Michael Crump has a done a nice tutorial on using them here.

I want to use them slightly differently though. In both the original use in the Coding4Fun toolkit and Michael’s post they place the button’s statically in the Window and have a cursor move over the top of them to Click them. I am going to reverse this event flow and make the Button itself the cursor and fire the Hovering event inside the control whenever the button is sitting over the top of one of my buttons.

To achieve this i am going to download the Coding4Fun Kinect Toolkit, add a reference to the library Coding4Fun.Kinect.Wpf and add a hover button to my MainWindow.xaml.

At the same time I’m going to setup my MainWindow.xaml with some new properties such as a background color, bigger screen size (1680x1045 – if your screen size is smaller you may have to change this to suit), and remove the navigation bars to make it run in full screen mode.

...

Project Information URL: http://www.diaryofaninja.com/blog/2011/10/19/remaking-the-xbox-kinect-hub--an-introduction-to-new-user-intErfaces

Project Download URL: http://www.diaryofaninja.com/asset/examples/kinectbuttonpress1.zip

Project Source URL: http://www.diaryofaninja.com/asset/examples/kinectbuttonpress1.zip

image

void SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
    if (e.SkeletonFrame.Skeletons.Count() == 0) return;

    SkeletonFrame skeletonSet = e.SkeletonFrame;

    SkeletonData firstPerson = (from s in skeletonSet.Skeletons
                                where s.TrackingState == SkeletonTrackingState.Tracked
                                orderby s.UserIndex descending
                                select s).FirstOrDefault();

    if (firstPerson==null) return;

    JointsCollection joints =  firstPerson.Joints;

    Joint rightHand = joints[JointID.HandRight];
    Joint leftHand = joints[JointID.HandLeft];

    //find which hand is being used for cursor - is the user right handed or left handed?
    var joinCursorHand = (rightHand.Position.Y > leftHand.Position.Y)
                    ? rightHand
                    : leftHand;

    float posX = joinCursorHand.ScaleTo((int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight).Position.X;
    float posY = joinCursorHand.ScaleTo((int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight).Position.Y;

    Joint scaledCursorJoint = new Joint
                            {
                                TrackingState = JointTrackingState.Tracked,
                                Position = new Microsoft.Research.Kinect.Nui.Vector
                                             {
                                                 X = posX,
                                                 Y = posY,
                                                 Z = joinCursorHand.Position.Z
                                             }
                            };
    OnButtonLocationChanged(kinectButton, buttons, (int)scaledCursorJoint.Position.X, (int)scaledCursorJoint.Position.Y);
}
private static void OnButtonLocationChanged(HoverButton hand, List<Button> buttons, int X, int Y)
{
    if (IsButtonOverObject(hand, buttons)) hand.Hovering(); else hand.Release();

    Canvas.SetLeft(hand, X - (hand.ActualWidth / 2));
    Canvas.SetTop(hand, Y - (hand.ActualHeight / 2));
}
void kinectButton_Clicked(object sender, RoutedEventArgs e)
{
    //call the click event of the selected button
    _selectedButton.RaiseEvent(new RoutedEventArgs(ButtonBase.ClickEvent, _selectedButton));
}
public static bool IsButtonOverObject(FrameworkElement hand, List<Button> buttons)
{
    if (_isClosing || !Window.GetWindow(hand).IsActive) return false;

    // get the location of the top left of the hand and then use it to find the middle of the hand
    var handTopLeft = new Point(Canvas.GetTop(hand), Canvas.GetLeft(hand));
    _handLeft = handTopLeft.X + (hand.ActualWidth / 2);
    _handTop = handTopLeft.Y + (hand.ActualHeight / 2);

    foreach (Button target in buttons)
    {
        Point targetTopLeft = target.PointToScreen(new Point());
        if (_handTop > targetTopLeft.X
            && _handTop < targetTopLeft.X + target.ActualWidth
            && _handLeft > targetTopLeft.Y
            && _handLeft < targetTopLeft.Y + target.ActualHeight)
        {
            _selectedButton = target;
           return true;
        }
    }
    return false;
}

Contact Information:

Follow the Discussion

  • Ian WalkerIan2 In geeks we trust ...

    Great post, thank you.

    Hoping / imagining that at some stage I will be able to go into VS and start a new Kinect project of type menu and it will give me boiler plate code that is along these lines.

  • Greg Duncangduncan411 It's amazing what a professional photographer can do...

    @Ian2: Check out, Kinect Contrib. Smiley

  • KiwieKiwie

    Great post. Just a noob's question: how this will work with navigating across different WPF pages with their own buttons. Do I need to repeat the codes for initializing the runtime object and the OnButtonLocationChanged method in each page?

  • Great job! Smiley

  • Great post. Just a noob's question: how this will work with navigating across different WPF pages with their own buttons. Do I need to repeat the codes for initializing the runtime object and the OnButtonLocationChanged method in each page?

     

    yea...i also wondering...how to do that?

  • @gduncan411: dear duncan,

    what is the algorithm of this?

    what is the methodology?

    thanks

  • Greg Duncangduncan411 It's amazing what a professional photographer can do...

    @kendrick0772: The best bet would be to contact the Project's author via their blog, http://www.diaryofaninja.com, or via twitter @dougrathbone. That will let you get the answers to your questions from the horse's mouth, so to speak...

  • @gduncan411:

    hi, thanks for the reply! Smiley

    sorry for broken English.

    and I still got questions to ask...

    Q: how this will work with navigating across different WPF pages with the buttons?

    for example :

    this following code is to close the window.

    private void buttonQuit_Click(object sender, RoutedEventArgs args)
    {
    Close();
    }

    but how about changing from window1 back to the mainwindow. after execute this.

    private void buttonQuit_Click(object sender, RoutedEventArgs args)
    
    {
    //what code should enter here???
    }

     

    link two pages...back to page1 (from page2)...go to page2 (from page1)

  • hi all!

    @dougrathbone

    this project is using SDK beta. but now SDK V1 released.

    May i know, where can i get the this project with SDK V1 version?

    did anyone have it?

    thanks Smiley

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.