Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Laser Tracking Camera

 In my last article, we saw how a stepper motor could be controlled with a computer by connecting it to its parallel port. This article will extend it to build laser following camera panner, which tracks and turns towards any laser dot in the camera's field of view. I'm assuming that you have already built a computer controlled stepper motor. If not, go through my previous article for information on how to build one. After you successfully build your laser following camera, you could use your own ideas and creativity to build even cooler things. For example, you could extend this project and build a cool laser following robot, or maybe even an autonomous laser guided turret which shoots down targets you point at!
Ashish's blog

Difficulty: Intermediate
Time Required: Less than 1 hour
Cost: Less Than $50
Software: Visual C# Express Edition
Hardware: A webcam, a laser pointer, a 5-wire unipolar stepper motor (these could also be salvaged from old 5¼" floppy disk drives), ULN2003 IC (stepper motor driver), wire, stripboard (or a solderless breadboard ), solder and DB-25 Male connector (buy these two if you can solder. Soldering is not necessary for doing this project, but it will ensure that your connections are secure), DB25 (female/male) parallel port cable, a multimeter, a power adapter (with voltage rating depending on your motor's requirements)
Download: Download solution
Video:
 

Mounting your webcam on the motor
I have simply mounted my webcam on a plastic cap. I used a heated nail for poking a hole through the cap's center:

Then, I fixed it on the motor by pushing the motor shaft through the hole:

I taped my webcam on the cap..and that was it.

My setup looks very sloppy, and I'm sure yours will look much better than mine. Smiley Anyway, let's move on to the fun part!

Finding the laser dot...


Our code will locate the laser dot by finding the brightest pixel in the webcam's field of view. If something in its view is brighter than the laser, the camera will simply turn towards that point instead of the laser dot. Smiley The program captures images from a webcam, using Andrew Kirillov's motion detection code for image acquisition. Then, it scans the pixels in the bitmaps captured. Based on their respective RGB (Red Green Blue) values, it determines how bright they are. We could use the Bitmap.GetPixel() function for accessing pixels and calculating the brightness of any pixel in a bitmap:

using System.Drawing;
 
Color c = someBitmap.GetPixel(x,y);
float b = c.GetBrightness();

Wow, that's easy! This code was simple to write, and easy to understand. However, unfortunately, it is VERY slow. If you use this code, it might take several seconds to process a single image becase the GetPixel()/SetPixel() methods are too slow for iterating through bitmaps of any real size. So, in this project, we'll make use of the BitmapData class in GDI+ to access the information we want. The BitmapData only allows us to access the data it stores through a pointer. This means that we'll have to use the unsafe keyword to scope the block of code which accesses the data. Based on an article by Eric Gunnerson, here's a class which will perform very quick unsafe image processing:

   1: public unsafe class UnsafeBitmap
   2: {
   3:     Bitmap bitmap;
   4:  
   5:     int width;
   6:     BitmapData bitmapData = null;
   7:     Byte* pBase = null;
   8:  
   9:     public UnsafeBitmap(Bitmap bitmap)
  10:     {
  11:         this.bitmap = new Bitmap(bitmap);
  12:     }
  13:  
  14:     public UnsafeBitmap(int width, int height)
  15:     {
  16:         this.bitmap = new Bitmap(width, height, PixelFormat.Format24bppRgb);
  17:     }
  18:  
  19:     public void Dispose()
  20:     {
  21:         bitmap.Dispose();
  22:     }
  23:  
  24:     public Bitmap Bitmap
  25:     {
  26:         get
  27:         {
  28:             return (bitmap);
  29:         }
  30:     }
  31:  
  32:     public struct PixelData
  33:     {
  34:         public byte blue;
  35:         public byte green;
  36:         public byte red;
  37:     }
  38:  
  39:     private Point PixelSize
  40:     {
  41:         get
  42:         {
  43:             GraphicsUnit unit = GraphicsUnit.Pixel;
  44:             RectangleF bounds = bitmap.GetBounds(ref unit);
  45:  
  46:             return new Point((int)bounds.Width, (int)bounds.Height);
  47:         }
  48:     }
  49:  
  50:     public void LockBitmap()
  51:     {
  52:         GraphicsUnit unit = GraphicsUnit.Pixel;
  53:         RectangleF boundsF = bitmap.GetBounds(ref unit);
  54:         Rectangle bounds = new Rectangle((int)boundsF.X,
  55:       (int)boundsF.Y,
  56:       (int)boundsF.Width,
  57:       (int)boundsF.Height);
  58:  
  59:         width = (int)boundsF.Width * sizeof(PixelData);
  60:         if (width % 4 != 0)
  61:         {
  62:             width = 4 * (width / 4 + 1);
  63:         }
  64:         bitmapData =
  65:       bitmap.LockBits(bounds, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
  66:  
  67:         pBase = (Byte*)bitmapData.Scan0.ToPointer();
  68:     }
  69:  
  70:     public PixelData GetPixel(int x, int y)
  71:     {
  72:         PixelData returnValue = *PixelAt(x, y);
  73:         return returnValue;
  74:     }
  75:  
  76:     public void SetPixel(int x, int y, PixelData colour)
  77:     {
  78:         PixelData* pixel = PixelAt(x, y);
  79:         *pixel = colour;
  80:     }
  81:  
  82:     public void UnlockBitmap()
  83:     {
  84:         bitmap.UnlockBits(bitmapData);
  85:         bitmapData = null;
  86:         pBase = null;
  87:     }
  88:     public PixelData* PixelAt(int x, int y)
  89:     {
  90:         return (PixelData*)(pBase + y * width + x * sizeof(PixelData));
  91:     }
  92: }

Be sure to check Eric's article on unsafe image processing.
This class can be used for retrieving the red, green and blue values of any pixel as shown below:

UnsafeBitmap uBitmap = new UnsafeBitmap (original_bitmap);
uBitmap.LockBitmap();
PixelData pixel = uBitmap.GetPixel (x,y);
uBitmap.UnlockBitmap();

You can also alter the red, green and blue values of any pixel in your bitmap:

UnsafeBitmap.PixelData pData = new UnsafeBitmap.PixelData();
pData.red = value;
pData.green = value;
pData.blue = value;
uBitmap.SetPixel (x, y, pData);

To find a laser dot in a bitmap, we have to go through the pixels in the bitmap and calculate each pixel's brightness to find the brightest one. Based on a pixel's RGB values, you can calculate brightness using this formula:

Brightness = (299 * red + 587 * green + 114 * blue) / 1000

This would give a value between 0 to 255.
Here's a sample code to check the brightness of each pixel in a 320 x 240 bitmap:

for (int y = 0; y < 240; y++)
{ 
    for (int x = 0; x < 320; x++) 
    {
        byte red, green, blue;
        red = uBitmap.GetPixel(x, y).red;
        green = uBitmap.GetPixel(x, y).green;
        blue = uBitmap.GetPixel(x, y).blue;
 
        float brightness = (299 * red + 587 * green + 114 * blue) / 1000;
 
        if (brightness > certainValue)
            // Do something
    }
}

After our program has the x and y coordinates of the laser dot, it can calculate the number of steps required to turn the webcam towards that point.

Using the Software
Since the program searches for the brightest pixel in the camera's field of view, the lighting conditions of your room can affect its performance. So, adjust the brightness threshold and lighting conditions so that nothing (except the laser) exceeds the brightness threshold.

If your stepper motor doesn't align your webcam with the laser dot properly, try adjusting the pixels-per-step track bar. It tells the program how much your stepper motor moves with each step. My stepper does 40 pixels/step (20 pixels while half stepping). Even though this isn't necessary, you can actually measure how many pixels your stepper moves in one step. Just comment out the calls to the MoveStepper() event in MotionDetector1.cs and run the program. Keep a laser dot fixed somewhere in your webcam's field of view and check its x coordinate in the Output window. After that, move the stepper one step in any direction with your mouse wheel, and see how much its x coordinate changes. Read my previous article for information on how to move a stepper with a mouse wheel, or simply download the code for doing it from here. Well, I still think it's not necessary that you actually measure how much your stepper moves using this tedious technique. Just play with the pixels-per-step track bar until you get things working perfectly.


Now what?

Now that you can track laser, and have a camera actually follow it, here are some interesting ideas to try:

  • Wire two stepper motors to your computer. Then, you could make a dual axis, pan/tilt assembly and have your camera follow laser dots in any direction.
  • Use two lasers and make an object tracking camera panner, which makes use of laser rangefinding to track the edge of any moving object.

Conclusion
We've reached end of this article. I hope you enjoyed reading it. Use your imagination and ideas to extend this project. If you come up with something interesting, I'd love to hear about it. Smiley Happy coding.

About the Author

Ashish Derhgawen is an IT student, currently living in New Delhi, India. He has been coding since fourth grade. Some of his other interests are harmonica playing, wildlife and cricket. When he's not at school, he spends his time working on unusual projects related to robotics, webcams, and electronics besides others. You can reach Ashish through his blog at http://ashishrd.blogspot.com.

Filed under: hardware hacks, robotics


Follow the Discussion

  • ColinColin

    You can avoid the /unsafe by using

    System.Runtime.InteropServices.Marshal.Copy to copy the image data into a byte array. There is an example in this project : http://www.codeproject.com/KB/GDI-plus/BitonalImageConverter.aspx

  • stephanestephane

    Why using a threshold ?

    You can just find the point where it's max !

  • Brian S. BrumfieldBrian S. Brumfield

    The threshold helps calibrate the camera to the background light conditions.

    Idealistically there should be a way to calibrate the detector to the laser color, and filter-out the non-laser hot spots. The real challenge is that red (like the red laser) is a component of white light, so you can't just look for the hottest red spot because the hottest white spot is likely to also be the hottest red spot. In the event that there is bright white light and or a warm background, this becomes challenging.

    A fuzzy algorithm "Where hotspot is mostly red and the pixels surrounding the hot spot are not" would be ideal.

    Nice work Ashish!

  • AshishAshish

    Hi Stephane,

    Without a threshold, the program would always find something "bright"..even when there's no laser dot in its view. Smiley

    Brian: That's a nice idea. I just didn't do it because I wanted to keep things simple. Smiley

  • jollywalker(glacaran@yahoo.com)jollywalker​(glacaran@y​ahoo.com)

    good day sir!

    may i ask if you succeeded in controlling the mouse movements by using the coordinates of your laser tracking project?

    thanx for your reply..

  • LeelaLeela

    Hi Ashish ,

    Your project is very good. I found the code for controlling a stepper motor with mouse wheel but could not find out how did you control the stepper motor to move along the laser pointer.Can you please help me with and send the solution for this project if possible through email.Please find my email ID : leelavaree@gmail.com

  • meysammeysam

    hi

    if you have any code or program for laser tracking?

    please give me if you have any code or program for laser tracking?

    thanks

  • Clint RutkasClint I'm a "developer"

    @meysam:  This article has that code in it.  Or you might want to check out the AForge lib.

  • AzeCrazeAzeCraze

    Hello Ashish, I have been reading all your articles with great pleasure. I am looking for a method to control a spotlight with servos to follow performers on theater stage. Your method seems to be the most ideal I have come across yet.

    However, I am considering using Infra Red Laser as the point light source.....and IR camera....so that the laser is not visible to the audience

    How well do you think your software will cope with this method?

  • AdelAdel

    Brilliant idea for a security camera! Only thing is the laser light might be safety hazard for the people being tracked!

  • s.avinashs.avinash

    can you give me a brief idea abouthow to mount this system over a robot to have a laser tracking robot

  • Clint RutkasClint I'm a "developer"

    @s.avinash I'd use a proper pan-tilt.  www.trossenrobotics.com/.../3139-Pan-and-Tilt-Kit-with-Servos.aspx  They are pretty cheap and effective.  From here you'd create the mount to whatever device you're putting on it.

  • mansawymansawy

    it's nice &may be easy to try something like that!

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.