Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Everyone Loves Babies! Webcams and Motion Detection

  Interfacing AIC250 Network Camera using .Net. Application allows to take digital snaps and stream them using MotionJPEG technique and publish live video on website.
Scott Hanselman's Computer Zen

Difficulty: Intermediate
Time Required: 6-10 hours
Cost: $100-$200
Software: Visual Studio Express Editions, Simon Fell's TCPTrace
Hardware: Airlink AIC 250 Network Camera
Download: Download

 

Summary: In the belated seventh installment of the "Some Assembly Required" column, Scott Hanselman extends, with permission, Andrew Kirillov's Motion Detector to .NET 2.0 and ClickOnce. He also works around some of the quirks in the AirLink AIC250 Network Camera by tracing network traffic and using a little intuition.

The AIC250 Network Camera

Some Assembly Required has been out of circulation for a while, but I'm back. I hope you all didn't forget about me. My wife and I had a baby boy, Zenzo, just 12 weeks ago and he's been a handful. He's mostly sleeping through the night now and I'm back to work, so I'll get back into the rhythm of writing this column on a regular basic. Please do continue to send in suggestions on what kinds of gadgets I can hook up to .NET.

I wanted to be able to see Zenzo while I'm at work, so I evaluated a number of solutions. I could use Video Chat built into MSN Messenger or other chat tools, but it's kind of inconvenient to bring the baby into the computer room and I often have trouble getting the video to work through the firewall. Using video chat would also mean that both sides would have to negotiate the conversation. If I wanted to make things simpler for my wife (and increase the WAF—Wife Acceptance Factor—of the solution), I'd want to just call her at home and start the camera remotely with no effort on her part.

I wanted a camera that would have its own network connection and wouldn't require an attached computer. This means that the camera would have some kind of integrated internal Web Server.

Additionally, my parents live two hours out of town and might want to occasionally peek in on the baby, and I want to make the process as easy as possible on them. Finally, I might want to hook up some kind of motion monitor if the camera detects that the baby is thrashing in his sleep.

I looked around at a number of Network Cameras and since price was my background priority, I settled on the AIC250 Network Camera. I picked up the Wired Ethernet version for US$99, but I understand that a wireless version of the same camera can be found around the net for $99 as well.

Seting up the camera is trivial: plug it in and it gets an IP address via DHCP. It has an integrated Web Server that lets you manage the camera's settings via your browser. The way the web-based interface displays the video stream is interesting, though. It includes two options, a Java Applet or an ActiveX control. These widgets are built into the firmware of the camera and, while the camera's firmware can be updated via software direct from the manufacturer, the video can officially only be viewed in the web interface in one of these two ways.

Being the hacker-type that I am, I wanted to get access to the underlying video stream. It's great that the camera includes all the software you need built-in, but if I could get access to the stream I could do all sorts of fun stuff.

MotionJPEG

I used Simon Fell's TCPTrace to peek at the network traffic as I logged into the web interface and looked at the video using the built-in Java applet. I noticed that the traffic was a series of JPEGs, one after the other. It was an HTTP GET that never ends. The connection stayed open and spit out JPEGs as fast as I could take it, around 10-15 frames a second. I had never seen traffic like this. I'd assumed that the camera would produce AVIs or MPEG files. Then I realized that spitting out JPEGs would be the cheapest possible way to produce video because there'd be no licensing of Video Codecs (Compressor/Decompressor). A little Googling showed me that this technique was called MotionJPEG or MJPEG.

I think of MotionJPEG as a clever hack, with the emphasis on clever. It's not the MPEG (Motion Picture Experts Group) video that you may be familiar with. Instead, MotionJPEG is a stream of JPEG images, one after the other. There doesn't appear to be a formal specification, just varying implementations.

There is really no such standard as "motion JPEG" or "MJPEG" for video. Various vendors have applied JPEG to individual frames of a video sequence, and have called the renkinfgsult "M-JPEG". JPEG is designed for compressing either full-color or gray-scale images of natural, real-world scenes. [Review of Video Streaming]

MotionJPEG over HTTP uses the Content-Type header "multipart/x-mixed-replace" along with a configurable boundary. This means that the stream is made up of Multiple Parts (hence multipart) and each new frame should replace the previous frame (hence x-mixed-replace). This particular camera sends an HTTP Header like this:

Content-Type: multipart/x-mixed-replace;boundary=--video boundary--

The boundary can be anything the server chooses, in this case "--video boundary--" which is a particularly intuitive choice. After the boundary there are additional headers including the actual Content-Type of the part, in this case, image/jpeg.

I started writing a parser/renderer from scratch in .NET 2.0 and the built-in WebClient libraries. It would be fairly easy to read the data into a buffer and watch for the boundary to go by, and then use the System.Drawing libraries to read the JPEGs in and render them as came in. However, I figured that there might be a library out there that would already do the work for me. That's when I found Andrew Kirillov's CodeProject article on Motion Detection.

The MJPEG handling in Andrew's is actually secondary to his code. His .NET 1.1 project is a showcase for his motion detection algorithms that we'll talk about in a moment, but I was interested in his pluggable VideoSource implementation.

Andrew's Implementation

Andrew created an interface called IVideoSource. He includes a number of concrete implementations, one being MJPEGSourcem the one I needed.

Visual C#

public interface IVideoSource
{
  event CameraEventHandler NewFrame;
  string VideoSource{get; set;}
  string Login{get; set;}
  string Password{get; set;}
  int BytesReceived{get;}
  object UserData{get; set;}
  bool Running{get;}
  void Start();
  void SignalToStop();
  void WaitForStop();
  void Stop();
  
}

The most inspired aspect of Andrew's implementation, the one I wouldn't have thought of myself, is the CameraEventHandler.

Visual C#

public delegate void CameraEventHandler(object sender, CameraEventArgs e);
public class CameraEventArgs : EventArgs
{
  private System.Drawing.Bitmap bmp;
  public CameraEventArgs(System.Drawing.Bitmap bmp)
  {
    this.bmp = bmp;
  }
  public System.Drawing.Bitmap Bitmap
  {
    get { return bmp; }
  }

}

The VideoSource throws each frame it finds as bitmaps packaged within an event. This might seem inefficient at first glance, but remember that this event would likely not fire more than 30 times a second, more likely around 5-10 times a second. Additionally, he's throwing a reference to a bitmap, so it's not like copies of bitmaps are flying around the system. Using eventing in this manner not only makes for a clean interface and separation between an IVideoSource implementation and the form that chooses to render it, but it also allows Andrew to insert his post-processing filters and motion detection algorithm.

Working Around Quirks of the AIC250

I had hoped that I'd be able to plug the Motion application directly into the AIC250 camera's MJPEG source. That turned out to be wishful thinking. There weren't any problems with the MJPEG implementation in Andrew's code, but there were a few quirks in the AIC250's embedded Web Server's handling of authentication.

(Note the misspelled "Auther" (sic) HTTPHeader that tells us that Steven Wu wrote this Web Server that reports itself as Camera Web Server/1.0.)

The AIC250 Camera supports HTTP Basic Authorization. When you visit the administration home page for the first time, you're prompted for a name and password using HTTP Basic Auth, then the ActiveX control or Java Applet is on the next page. My TCP sniffing session earlier told me that the stream of JPEGs was the result of an HTTP GET to an endpoint called /mjpeg.cgi.

Click here for larger image

(click image to zoom)

I assumed that I could make a call to /mjpeg.cgi with .NET, passing in the name and password via HTTP Basic Auth and things would work great. It didn't work, though. However, if I "visited" the home page programmatically first, THEN requested /mjpeg.cgi, the MJPEG stream began. Apparently the Camera sets an internal flag after a user hits the home page that allows the next request to hit /mjpeg.cgi. I suspect this is because the /mjpeg.cgi endpoint is coded to simply return JPEGs, nothing else. They (Steven Wu?) likely decided that getting /mjpeg.cgi to support authentication and meet the HTTP spec exactly would have been a hassle and wouldn't have provided value. Their use case never suspected that some fool (me) would try to get to the video stream directly.

The next thing I ran into was that HTTP Basic Authentication is supposed to wait for an Authentication Challenge first before sending credentials. This web server seems to want the credentials sent right away. Fortunately the WebRequest class supports a PreAuthenticate property that sends the authentication credentials first, before the challenge.

I changed the MJPEGSource to support both "PreAuthentication" and "Authentication with the Home Page" options and I was off and running.

More Quirks - UnsafeHeaderParsing

That wasn't the end of the quirks I'd have to fight. After upgrading the application from .NET 1.1 to .NET 2.0, I started getting exceptions while calling HttpWebRequest.GetResponse(). However, the exceptions weren't very informative. The only thing that had changed was the upgrade from 1.1 to 2.0. A little Googling, MSDNing, and digging led me to Mike Flasko's Blog:

"By default, the .NET Framework strictly enforces RFC 2616 for URI parsing. Some server responses may include control characters in prohibited fields, which will cause the System.Net.HttpWebRequest.GetResponse method to throw a WebException. If useUnsafeHeaderParsing is set to true, System.Net.HttpWebRequest.GetResponse will not throw in this case; however, your application will be vulnerable to several forms of URI parsing attacks. The best solution is to change the server so that the response does not include control characters."

Of course, Steven Wu's built-in Web Server in this Camera isn't going to change, so I had little choice but to include an app.config like this:


<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.net>
    <settings>
      <httpWebRequest useUnsafeHeaderParsing="true" />
    </settings>
  </system.net>
</configuration>
Motion Detection

Andrew includes four different motion detection algorithms that he describes in detail in his CodeProject Article. His algorithms are focused on image processing, rather than video processing, which is why he throws each frame as a bitmap, saving the last frame and comparing it to the current frame. Not only can he plug in different video sources, since he "normalizes" all video sources to a series of bitmaps he can apply any motion detection algorithm to any video source.

In the image above, the teddy bear on the right has just been moved with a string. The red boxes around the bear were added by the motion detection algorithm indicating what portion of the image changed.

ClickOnce

At this point I had a WinForms 2.0 application that let me view the camera in my son's room. I forwarded a port from my cable modem's external IP address to the internal IP address of the camera so I could connect to it from outside or at work. The next step was to make the application a ClickOnce application so my parents or I could run it from anywhere.

Visual Studio makes creating a ClickOnce application very straightforward. Right-click on the project within the Visual Studio Solution Explorer and select Properties. First visit the Signing tab and create a Test Certificate (unless you have your own Signing Certificate). The certificate will be used to sign the ClickOnce manifest that will be published to the web site. Next visit the Security tab and select "This is a Full Trust application." If you create your own ClickOnce application, you can specific the exact permissions that your application needs to run. Finally, from the Publish tab you can indicate the publishing location and ultimate URL. I chose to publish to a "publish" folder, then upload the contents to my website manually because I'm a control freak.

Once the project is configured, you can also build and publish a ClickOnce application by running msbuild /target:publish from a Visual Studio command line.

As an interesting sidenote, I originally blogged about problems with ClickOnce and FireFox but a member of the ClickOnce team blogged an explanation. Fortunately this will be fixed in the next version of the Framework.

Now, anyone in my family can visit our private website, ClickOnce and run this .NET 2.0 WinForms application and connect with a name and password to the AIC250 Network Camera. Andrew's motion detection is an added bonus.

Conclusion

There are a number of fun things that could be extended, added, and improved on with this project. Here are some ideas to get you started:

  • Add your own video source that reads Animated GIFs, DivX or other formats.
  • Build a video source implementation that connects to WIA (Windows Image Acquisition) devices.
  • Hook up to other Network Cameras.
  • Create your own Motion Detection algorithm that compares to frames and see how it stacks up against Andrew's.
  • Make an event that fires on motion and plugins to react to that motion. Hook up lights, send emails, or whatever you like!
  • Extend the application to support multiple cameras (maybe up to 4) via a "split screen" feature.

Have fun and have no fear when faced with the words: Some Assembly Required!

If you do extend this application, be sure to release the source. Thanks again to Andrew Kirillov for allowing me to extend his wonderful source. My family thanks you.

Follow the Discussion

  • tiryakitiryaki

    nice work. i hope your baby is well Smiley

  • Gonzalo Romero AttonGonzalo Romero Atton

    Very interesting article. I was thinkin' if it is possible to hook each motion capture event to a camera rotation movement, of course in the direction the motion capture indicates there was movement.

    Anyway, great article. I'll try to buy some sort of web cam like the one you used. It's an interesting proyect to do.

    And lovely baby indeed, congrats.

    Regards,

    Gonzalo

  • SoftlionSoftlion

    Nice !

    I found your blog entry through Andrew Kirillov nd wanted to show you that at least one person read it Wink

  • valconvalcon

    hi scott. u have done a good job. I'm working on a project related to motion detection too.I'm a U student. And now i'm facing some problem. Can u teach me how to write C code to open a video file and decode it to frame so that i can run some motion detection on it?

    Please feel free to contact me. valyandre@gmail.com

    thanks. Appreciate for everything.

  • LJLJ

    Found this information on Andrew Kirillov's page. Nice work! Recently I've been looking into video manipulation. Most of my time has been spent on business applications and hardware interfacing. Great work on this project.

  • Michael LimanniMichael Limanni

    Is there an easy VB way to create an application that just has a URL and video image for video conferencing?  I would encode a user name and password into the application.  I want something that the client can launch without using the Java program in the Web browser.  I would just look more professional.

  • WolfgangWolfgang

    Interestingly enough I got to your page in search of the "Auther" Wink Steven Wu. The same header information is supplied with the Hawking Technology HNC300 camera, which has been discontinued. I am wondering how the AIC250 works, but assume that the viewing angle is very narrow as well due to the focal length.

    BTW: I am evaluating the ~ US$198 software package "Security Monitor Pro v2.2" from "DeskShare" to save myself from the trouble you have been going through. Works great. The one good thing to know is that the camera supplies the current image through /image.jpg.

    Good luck with your project!

  • IstvanIstvan

    I'm trying to access an Ip camera using your technique. But I'm having some problems. When I'm running the code the debugger is showing: "flushing" flushin"....and so on. Could you help me?

  • Clint RutkasClint I'm a "developer"

    @Istvan the flushing statement is debug information Scott put in.  I suggest hooking up a debugger rather than just viewing the output window.

  • ArvindArvind

    Thanks for the example.  I got to your site searching for a solution to my AirLink101 AICN500 IP camera.  I am able to view the camera through a browser using activex or Java but was not sure how to manage the stream that is output at /cgi/mjpg/mjpeg.cgi

    I tried with your code and I am not getting the display on the screen.  When I debug, the first issue I saw was that inside the Mjpegsource.cs, the following line was always evaluating to TRUE

    if (ct.IndexOf("multipart/x-mixed-replace") == -1)

            throw new ApplicationException("Invalid URL");

    I debugged and saw that what I was really getting was "multipart/mixed".   So I changed the code above to check for multipart/mixed and got around that issue.

    Then I see that I get an exception at the below line in MJpegSource.cs

    Bitmap bmp = (Bitmap) Bitmap.FromStream(new MemoryStream(buffer, start, stop - start));

    The exception is "Parameter is not valid."  This one has me stumped.  Any pointers as what the issue could be?

    Thanks.

  • usmanusman

    hi there

    your work has been of great help to me

    thankyou

    what i am trying to do here is that i am streaming from a web cam over a network and trying to capture that stream on this very thing ure using

    i aint got no idea how do i directly access a webcams streams using http without using vlc

    am i missing something

  • Clint RutkasClint I'm a "developer"

    @usman, a lot depends on how the stream is being sent to you.

  • Sensor_manSensor_man

    hi, i coding about Camera IP but i using C++ so can you convert your code from C# to C++ for me?

    I need code about Camera IP with Motion Jpeg.

    Can you help me? thanks!

  • Clint RutkasClint I'm a "developer"

    @Sensor_man sorry, this example was only done in c# and vb.net.  Porting should be pretty straight forward since you have access to the source code.

  • Clint RutkasClint I'm a "developer"

    @oop this code is from an IP based camera.  Getting access to a mjpeg data stream may be a bit hard.

  • oopoop

    hi, can anyone give me a public camera IP so that I can try this out before I buy an IP camera? Thanks

  • Clint RutkasClint I'm a "developer"

    @sudheer karumanchi  This project leverages mjpeg so this should work if you adjust the endpoint.

  • sudheer karumanchisudheer karumanchi

    Hi, this was an excellent project with ip cameras...thanks for providing these type of projects.

    i am also doing a similar project, to read mjpeg streams from an axis ip camera to a smartphone(windows or android). can you help in this context how far can i proceed to do so.

    thanks & regards,

    Karumanchi

  • DariofilDariofil

    www.axis.com/camerademo lots of cameras...

  • JaspreetJaspreet

    hi

    I want to read header of Ip Camera Video Stream.Can anybody tell me how we can read header of input stream of IP Cameras in java.

    Thanks

  • Clint RutkasClint I'm a "developer"

    @Jaspreet this is a blog that deals with .NET tech, sorry!  For an upcoming article, we read the stream from the ip camera and parse based off the incoming data.

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.