How about a little physics in your augmented reality?


Today's post we're revisiting Sliverlight 5 (beta), augmented reality and the SLARToolkit, courtesy an update from Rene Schulte.

When I read his post, I couldn't help but think of Will It Blend, where they keep throwing stuff into a blender to see if it can be blended... Like this project, where Rene seems to keep throwing more and more stuff into his SLARToolkit project. Today we're looking at how he's added Balder and JigLibX into the mix...

Cubelicious - Silverlight 5 + Balder + Physics + SLARToolkit Augmented Reality = Triple Win!

This post here provides a new demo for SLARToolkit which uses the open source 3D engine Balder by my friend Einar Ingebrigsten. This demo also leverages the open source physics engine JigLibX my buddy Andy Beaulieu ported over to Silverlight. You can try the live sample if you have the Silverlight 5 beta installed or watch a video instead.



A webcam and at least the Silverlight 5 beta runtime must be installed to run the sample. It's available here. Alternatively there is also a video of the new sample embedded below.

If you want to try it yourself you need do download the SLAR and / or L marker, print them and hold them in front of the camera. The marker(s) should be printed non-scaled at the original size (80 x 80 mm) and centered for a small white border. As an alternative it's also possible to open a marker file on a mobile device and to use the device's screen as marker. Also make sure the camera is set up properly and the scene is illuminated well without hard shadows. See the SLARToolkit Markers documentation for more details.

Open the sample in a new page.


How it works

This sample uses the webcam video stream which fills a Rectangle shape, the video stream is also constantly captured and fed to the SLARToolkit BitmapMarkerDetector to detect the markers. The detection result contains a transformation matrix for each found marker which is then used to apply a global transformation to the cubes and the plane.

I implemented a particle system with a flexible directed emitter which can be controlled through various properties. The particle system is quite generic and can be used for all kinds of particles (3D objects). The particle collision detection and resolving is based on rigid body physics that was implemented with the help of the JigLibX library my Silverlight MVP buddy Andy Beaulieu ported over to Silverlight.

The rendering and the model loading is done by the 3D engine Balder. It's a fantastic open source engine by Einar Ingebrigtsen. You just need to write a couple lines of XAML and you're good to go. This sample only uses a simple cube model, but Balder has built-in model loaders to load complex 3D models and Einar provides a big sample library. He also brought the engine to a few more platforms like Windows Phone 7, OpenGL and has even a neat software rendering fallback. Read his blog post here.

As part of this sample I needed some vector and quaternion methods which were missing in Balder. I contributed those and the generic particle system to the Balder project. Feel free to use the particle system and the other methods in your Balder projects.

Let's peek at the Project...


private void DetectMarkers(WriteableBitmap bmp)
   // Init. here because the captureSource.VideoCaptureDevice.DesiredFormat getter is not reliable and throws an Exception
   if (!isInit)
      // Init AR
      arDetector.Initialize(bmp.PixelWidth, bmp.PixelHeight, Game.Camera.Near, Game.Camera.Far, slarMarker);
      isInit = true;

   // Detect
   var dr = arDetector.DetectAllMarkers(bmp);

   // Reused for marker highlighting
   ViewportOverlay.Source = bmp;

   if (dr.HasResults)
      var transformation = Balder.Math.Matrix.CreateTranslation(0, -5, 0) * Balder.Math.Matrix.CreateRotationX(90) * dr[0].Transformation.ToBalder();

      // Highlight detected markers
      var txt = String.Empty;
      foreach (var r in dr)
         bmp.DrawQuad((int)r.Square.P1.X, (int)r.Square.P1.Y, (int)r.Square.P2.X, (int)r.Square.P2.Y, (int)r.Square.P3.X, (int)r.Square.P3.Y, (int)r.Square.P4.X, (int)r.Square.P4.Y, Colors.Red);
         txt += String.Format("{0}.Confidence = {1:0.00}   ", r.Marker.Name, r.Confidence);

using Balder;
using Balder.Execution;
using Balder.Materials;
using Balder.Math;
using JigLibX.Collision;
using JigLibX.Physics;

namespace SLARToolKitBalderSampleSL5
   public class SlarGame : Game
      const float UpdateStep = 0.04f;

      readonly PhysicsSystem world;
      readonly Cuboid ground;

      public SlarParticleSystem ParticleSystem { get; private set; }

      public SlarGame()
         ParticleSystem = new SlarParticleSystem();

         world = new PhysicsSystem
                    CollisionSystem = new CollisionSystemSAP { UseSweepTests = true },
                    EnableFreezing = true,
                    SolverType = PhysicsSystem.Solver.Normal,
                    NumCollisionIterations = 8,
                    NumContactIterations = 8,
                    NumPenetrationRelaxtionTimesteps = 15

         ground = new Cuboid(new Vector(300, 1, 300))
                     Material = new Material {Diffuse = new Color(50, 50, 50, 100), DoubleSided = true},
                     IsFixed = true,

 Here’s a few more links you might find interesting:

The Discussion

  • User profile image

    Very exciting seeing this kind of stuff - really does hold lots of promise for the future (in terms of augmented reality gaming)

  • User profile image
    Shayne Speares

    The rendering and the model loading is done by the 3D engine Balder. It's a fantastic open source engine by Einar Ingebrigtsen.

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.