Camera Fundamentals

Download this episode

Download Video

Description

In the Camera Fundamentals Quickstart video, you’ll learn:

  • How to leverage the AllFramesReady event to synchronize data between the color, depth, and skeletal frames
  • The supported resolutions for the color and depth cameras
  • How to read the color byte array from the ColorImageFrame event and how to dynamically build a Bitmapsource to display the color image
  • How to use the KinectColorViewer user control to display the Kinect’s color image simply by databinding the Kinect property to the KinectSensorChooser user control
  • How to tilt the Kinect camera and how it determines it’s current angle

Resources

Embed

Format

Available formats for this video:

Actual format may change based on video formats available and browser capability.

    The Discussion

    • User profile image
      Jazon

      Dan rules!

    • User profile image
      joze

      Great tutorial. Keep up the good work!

    • User profile image
      El Bruno

      Great demo Dan !!! 

      Thanks Big Smile

    • User profile image
      Adam Nuttall

      Useful introductory videos - thanks Dan.

    • User profile image
      Ethan

      Thanks Dan. Your videos have been so helpful!

    • User profile image
      Zap

      This series has been super helpful. Really awesome.

    • User profile image
      PixelPark

      Thanks for the update!

      Perhaps better use:
      int stride = colorFrame.BytesPerPixel * colorFrame.Width;

    • User profile image
      Reuben

      Thanks pixelpark. RGB plus Alpha transparency makes sense, what does colorFrame.Width truly represent?

    • User profile image
      Eric

      Amazing tutorial very helpful, many many thanks Dan!

    • User profile image
      Antonis

      Guys am kinda new to visual c#, and i get an error System.Windows.Media.PixelFormat does not contain a definition for 'Bgr32'.
      The error is on PixelFormat.Bgr32, anyone knows how can i fix this?

    • User profile image
      orc536

      studying...

    • User profile image
      atsprink

      I am new to all of these languages and I am developing with the Kinect for my senior design so I decided to look at these videos to start out.  How come when I combine the code from 2.Setting up the Environment and 3. Camera Fundamentals, I no longer get the camera to display an image.   I have the following code which is just basically the code from video 2 and 3 combined.  If I use the bitmap and byte[] i can get the image but not if I change it like the author of the video does.  Can someone help me?

       

      
      using System;
      using System.Collections.Generic;
      using System.Linq;
      using System.Text;
      using System.Windows;
      using System.Windows.Controls;
      using System.Windows.Data;
      using System.Windows.Documents;
      using System.Windows.Input;
      using System.Windows.Media;
      using System.Windows.Media.Imaging;
      using System.Windows.Navigation;
      using System.Windows.Shapes;
      using Microsoft.Kinect;
      using Coding4Fun.Kinect.Wpf;
      
      
      
      namespace Gesture_Based_Controller
      {
          /// <summary>
          /// Interaction logic for MainWindow.xaml
          /// </summary>
          public partial class MainWindow : Window
          {
              //this is a *template* app that includes library references only. It does not do anything but use SensorChooser
              //to manage Kinect state
      
              public MainWindow()
              {
                  InitializeComponent();
              }
      
              private void Window_Loaded(object sender, RoutedEventArgs e)
              {
                  //SetupKinectManually(); 
                  kinectSensorChooser1.KinectSensorChanged +=new DependencyPropertyChangedEventHandler(kinectSensorChooser1_KinectSensorChanged);
              }
      
              private void kinectSensorChooser1_KinectSensorChanged(object sender, DependencyPropertyChangedEventArgs e)
              {
                  var oldSensor = (KinectSensor)e.OldValue;
      
                  // stop old sensor
                  if (oldSensor != null)
                  {
                      StopKinect(oldSensor);
                  }
      
                 var newSensor = (KinectSensor)e.NewValue;
                 if (newSensor == null)
                 {
                      return;
                 }
      
                  // turn on features that we need
                  newSensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
                  newSensor.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30);
                  newSensor.SkeletonStream.Enable();
      
      
                  try
                  {
                      newSensor.Start();
                  }
                  catch (System.IO.IOException)
                  {
                      //another app is using Kinect
                      kinectSensorChooser1.AppConflictOccurred();
                  }
              }
      
              // this event fires when Color/Depth/Skeleton are synchronized
             /* void newSensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)
              {
                  using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
                  {
                      if (colorFrame == null)
                      {
                          return;
                      }
      
                      byte[] pixels = new byte[colorFrame.PixelDataLength];
                      colorFrame.CopyPixelDataTo(pixels);
      
                      // how many bytes do we need on a per row basis  bgr 32 b,g,r, empty, b, g,r, empty
                      int stride = colorFrame.Width * 4;
                      image1.Source = BitmapSource.Create(colorFrame.Width, colorFrame.Height, 96, 96, PixelFormats.Bgr32, null, pixels, stride);
      
                  } 
              }*/
      
              private void StopKinect(KinectSensor sensor)
              {
                  if (sensor != null)
                  {
                      if (sensor.IsRunning)
                      {
                          //stop sensor 
                          sensor.Stop();
      
                          //stop audio if not null
                          if (sensor.AudioSource != null)
                          {
                              sensor.AudioSource.Stop();
                          }
      
      
                      }
                  }
              }
      
              private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
              {
                  StopKinect(kinectSensorChooser1.Kinect);
              }
          }
      }
      
      

       

       

       

    • User profile image
      Dan

      Guys am kinda new to visual c#, and i get an error System.Windows.Media.PixelFormat does not contain a definition for 'Bgr32'.
      The error is on PixelFormat.Bgr32, anyone knows how can i fix this?

      One thing to remember is that formats is plural as in PixelFormats.Bgr32. It's a subtle difference for sure, but I suspect that's your issue. Here's a snippet from the Camera Fundamentals Quickstart and note the plurality in formats.

                         

      BitmapSource.Create(colorFrame.Width, colorFrame.Height,96, 96, PixelFormats.Bgr32, null, pixels, stride);
       

       

    • User profile image
      Dan

      , atsprink wrote

      I am new to all of these languages and I am developing with the Kinect for my senior design so I decided to look at these videos to start out.  How come when I combine the code from 2.Setting up the Environment and 3. Camera Fundamentals, I no longer get the camera to display an image.   I have the following code which is just basically the code from video 2 and 3 combined.  If I use the bitmap and byte[] i can get the image but not if I change it like the author of the video does.  Can someone help me?

      In looking at your code, if you are trying to get AllFramesReady to fire, you have to do two things

      1. Tell the sensor you want to register when all the frames are ready by subscribing to the AllFramesReady event and telling it what method you want to execute, which is the newSensor_AllFramesReady event already in your code. Without this, the Kinect sensor doesn't know to tell you that an event has occured

      newSensor.AllFramesReady +=newSensor_AllFramesReady;

      2. You need to uncomment out the newSensor_AllFramesReady event. In the example below, the event is commented out (using /* and */ ) . Since this event is commented out (it will be all green in Visual Studio), it never executes. Remove the /* and */ and that event should execute assuming you have followed step #1.

       

         /* void newSensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)        {            using (ColorImageFrame colorFrame = e.OpenColorImageFrame())            {                if (colorFrame == null)                {                    return;                }                 byte[] pixels = new byte[colorFrame.PixelDataLength];                colorFrame.CopyPixelDataTo(pixels);                 // how many bytes do we need on a per row basis  bgr 32 b,g,r, empty, b, g,r, empty                int stride = colorFrame.Width * 4;                image1.Source = BitmapSource.Create(colorFrame.Width, colorFrame.Height, 96, 96, PixelFormats.Bgr32, null, pixels, stride);             }         }*/
    • User profile image
      Duncanma

                  //using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
                  //{
                  //    if (colorFrame == null)
                  //    {
                  //        return;
                  //    }
      
                  //    byte[] pixels = new byte[colorFrame.PixelDataLength];
                  //    colorFrame.CopyPixelDataTo(pixels);
      
                  //    int stride = colorFrame.Width * 4;
                  //    image1.Source =
                  //        BitmapSource.Create(colorFrame.Width, colorFrame.Height,
                  //        96, 96, PixelFormats.Bgr32, null, pixels, stride);
      

    • User profile image
      Aharris

      I'm pretty new at this and have been following the videos. While extremely helpful I've come to an issue I can't resolve. No matter what I try the kinect would display the RGB Image

      I've followed both parts 2 and 3 but for some reason it still wont display my image.

       

      namespace WpfApplication1
      {
          /// <summary>
          /// Interaction logic for MainWindow.xaml
          /// </summary>
          public partial class MainWindow : Window
          {
              public MainWindow()
              {
                  InitializeComponent();
              }
      
              private void Window_Loaded(object sender, RoutedEventArgs e)
              {
                  kinectSensorChooser1.KinectSensorChanged += new DependencyPropertyChangedEventHandler(kinectSensorChooser1_KinectSensorChanged);
      
              }
      
              void kinectSensorChooser1_KinectSensorChanged(object sender, DependencyPropertyChangedEventArgs e)
              {
                  var oldSensor = (KinectSensor)e.OldValue;
      
                  if (oldSensor != null)
                  {
                      StopKinect(oldSensor);
                  }
                  var newSensor = (KinectSensor)e.NewValue;
                  if (newSensor == null)
                  {
                      return;
                  }
      
                  newSensor.AllFramesReady += new EventHandler<AllFramesReadyEventArgs>(newSensor_AllFramesReady);
      
                  // turn on features that you need
                  newSensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
                  newSensor.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30);
                  newSensor.SkeletonStream.Enable();
                  
                  try
                  {
                      newSensor.Start();
                  }
                  catch (System.IO.IOException)
                  {
                      //This happens if another app is using the kinect
                      kinectSensorChooser1.AppConflictOccurred();
      
                  }
              }
      
              void newSensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)
              {
                  using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
                  {
                      if (colorFrame == null)
                      {
                          return;
                      }
      
                      byte[] pixels = new byte[colorFrame.PixelDataLength];
                      colorFrame.CopyPixelDataTo(pixels);
      
                      int stride = colorFrame.Width * 4;
                      image1.Source =
                          BitmapSource.Create(colorFrame.Width, colorFrame.Height,
                          96, 96, PixelFormats.Bgr32, null, pixels, stride);
                  }
      
              }
              void StopKinect(KinectSensor sensor)
              {
                  if (sensor != null)
                  {
                      sensor.Stop();
                      sensor.AudioSource.Stop();
                  }
              }
      
              private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
              {
                  StopKinect(kinectSensorChooser1.Kinect);
      
              }
          }
      }

    • User profile image
      owh

      I tried to include the code for elevation control and notice that when the kinect moves (tilts), the image does not follow. The image gets updated only when the kinect completed its tilt. Is there anyway to have the image being updating while the kinect moves (tilts)? Thanks for the great tutorial and many helpful people giving solutions to problems.

    • User profile image
      Dan

      I don't know of any way to have image data while tilting as its not exposed in the API. One option would be to move just a little, scan, then move again.

      I tried to include the code for elevation control and notice that when the kinect moves (tilts), the image does not follow. The image gets updated only when the kinect completed its tilt. Is there anyway to have the image being updating while the kinect moves (tilts)? Thanks for the great tutorial and many helpful people giving solutions to problems.

    • User profile image
      owh

      @Dan, many thanks for the clarification.

    • User profile image
      kaido

      I see video coming from the camera, but nothing else, and the green light keeps flashing. Also the Kinect in sleep mode, meaning it's tilt down instead of raised up.

      Does anyone know what's wrong?

    • User profile image
      Dan

      @kaido:

      I see video coming from the camera, but nothing else, and the green light keeps flashing. Also the Kinect in sleep mode, meaning it's tilt down instead of raised up.

      Does anyone know what's wrong?

      If you see video, then the Kinect is working. Try using the samples that ship with the Kinect Developer toolkit, like Kinect Explorer. The download is available here http://www.microsoft.com/en-us/kinectforwindows/develop/developer-downloads.aspx

       

      The "tilt down" is something you, as a developer can control. Using the Kinect Explorer sample, set the Tilt to 0 and, assuming the Kinect is on a flat table, that should give you the correct angle.

       

    • User profile image
      Juan Pablo

      Hi Dan, I just don't seem to understand what is that "96" number in the BitmapSource.Create method. You said that it is dpiX and dpiY, but is it a strict value? If it is then why isn't it a Constant? And what if I change my video res to 1280 x 960, which would be the appropriate dpi?

      Thanks!

    • User profile image
      Mauricio

      Dan,

      Thanks you very much. Great introduction for newbies interested in Kinect.

      Regards,
      Mauricio

    • User profile image
      Lys

      Dan, could you please show de code that you used for testing the tilt?

      thanks

    • User profile image
      Fabrizio Borgia

      To anyone watching this video and developing with SDK 1.5+.
      You will encouter issues with the KinectColorViewer, things have changed a little since the SDK1.0 (the one that DAn is using).
      You can find very useful info on the following quiestions on StackOverflow.

      http://stackoverflow.com/questions/13256124/kinectcolorviewer-and-sdk-1-6

      http://stackoverflow.com/questions/12609858/using-kinectcolorviewer-in-sdk1-5

    Comments closed

    Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.