• PDC09: Code Visualization, UML, and DSLs

• Cameron’s Blog

Introduction

Back in July of 2009 I learned of Neurosky's MindSet device via Engadget.com.  The MindSet is a a brain-wave sensing headset costing only $200.  I contacted the company and was placed on a short list to receive hardware when available to build a demo for PDC 2009.  This is the result of that project.

MindBlaster is a pretty simple game written using XNA which runs on a Windows PC in conjunction with the MindSet headset and a Nintendo Wiimote.  The object of the game is to move your head, which moves a targeting reticule on the screen, to aim at enemy alien ships, then focus your concentration on the selected ship to heat it up and eventually make it explode.  The more you focus, the faster the alien ship explodes.  As more enemies are destroyed, they faster they fire back at you, causing a pretty frenetic gameplay experience until the onslaught becomes too much and you are destroyed.

So let's take a look at some of the components that went into creating MindBlaster and how the integrate into the final game.

MindSet Headset

As stated, the Mindset is a headset from Neurosky that is capable of reading alpha, beta, theta and gamma brain waves via 4 dry sensors, 3 located on the left earpiece, and one on an arm that touches the forehead.  In addition, the device contains a microphone and ear speakers, making it a full-on audio headset as well.

This device connects to a PC via Bluetooth and shows up as a Bluetooth serial port.  This means we can communicate with the device extremely easily.  However, Neurosky also includes their own API with a simple C# wrapper that makes .NET development even easier.  On top of that, I wrote a simple API which somewhat mimics my Managed Wiimote Library, which is available as a separate download from this application to make .NET development with the Mindset a bit easier.

One of the interesting things about the MindSet is that you don't actually need to know what brain-waves are or how they work.  I don't know the difference between an alpha wave and a theta wave, but I was still able to produce a decent game using brain waves.  Neurosky has developed a proprietary algorithm which takes your brain-wave values and turns them into a single score from 1 to 100 of “attention” and “meditation” allowing you can very easily determine how hard a person is concentrating or relaxing.

Using the provided API is pretty simple.  Here is a chunk of code that connects to the headset and pulls brain-wave values in a loop.  You wouldn't want to use this directly in your application, but it covers the important points:

// get a connection ID
int connectionId = ThinkGear.TG_GetNewConnectionId();
if(connectionId < 0)
    return false;

// connect to a COM port
int connect = ThinkGear.TG_Connect(_connectionId, @"\\.\COM1", baud, ThinkGear.STREAM_PACKETS);
if(connect < 0)
    return false;

while(true)
{
    // read data packets
    int packetsRead = ThinkGear.TG_ReadPackets(_connectionId, -1);
    if(packetsRead > 0)
    {
        // get the attention value
        if(ThinkGear.TG_GetValueStatus(_connectionId, ThinkGear.DATA_ATTENTION) != 0)
            int attention = ThinkGear.TG_GetValue(connectionId, ThinkGear.DATA_ATTENTION);

        // get the meditation (relaxation) value
        if(ThinkGear.TG_GetValueStatus(_connectionId, ThinkGear.DATA_MEDITATION) != 0)
            int attention = ThinkGear.TG_GetValue(connectionId, ThinkGear.DATA_MEDITATION);
    }
}

This opens a connection ID with the API, opens a connection to the headset on a provided COM port, then, in a loop, reads all available packets from the headset and retrieves the current state of the values requested.

I have created a very simple library which interfaces with the headset and put it up as a separate download.  If you're just interested in using this headset with .NET, please have a look at my ThinkGearNET library.

Wiimote

A Nintendo Wiimote is strapped upside-down to the top of the headset and its IR camera is used for 2D head tracking.  For PDC, I removed the Wiimote's guts from its case and mounted them directly to the headset, adding a battery holder box from Radio Shack.  This isn't really necessary, and you can just mount your own cased Wiimote on top.

The Wimote contains a 1024x768 camera which can detect the positions of up to 4 IR sources.  For this application, I only needed a single IR source since rotation is unimportant.  I used the Nyko Wireless Sensor Bar for this source with one side covered in electrical tape.

Using my Managed Wiimote Library, I am able to poll the current state of the IR camera to pull the XY position of the IR source and translate that to the position of the targeting reticule on the screen.

WiimoteState wmState = MindBlasterGame.Wiimote.WiimoteState;
if(wmState.IRState.IRSensors[0].Found)
{
    _reticule.Position.X = (wmState.IRState.IRSensors[0].Position.X) * MindBlasterGame.ScreenSize.X;
    _reticule.Position.Y = MindBlasterGame.ScreenSize.Y - (wmState.IRState.IRSensors[0].Position.Y) * MindBlasterGame.ScreenSize.Y;
}

Game Architecture

The game uses parts of the XNA Creators Club Game State Management sample for the overall game and screen architecture.  Each screen is broken out into its own object with its own Update, Draw, LoadContent, etc. methods.  This allows the title screen, settings screen, gameplay screen, and final scoring screen to remain separate “entities” for easier development.

Let's talk a little bit about each screen and important points for each.

Settings Screen

While developing at home and demonstrating this at PDC, I needed a very quick and dirty settings screen to easily enable and disable various features of the game.  For example, I really didn't want to wear the headset the entire time I was developing, so I wrote some code which allowed me to use the mouse in place of the actual hardware.  The settings screen allowed me to easily toggle those types of things.  This was accomplished by creating an object that inherits from the MenuScreen class from the Game State Management sample.  Menu entries can be then created as follows:

private readonly MenuEntry _menuEntry;
_menuEntry = new MenuEntry("My menu entry");
_menuEntry.Selected += _menuEntry_Selected;
MenuEntries.Add(_menuEntry);

This creates the entry, sets its event handler, and adds it to the list.  In the event handler, you can do whatever might be required; toggle an option on or off, increase or decrease a value, etc.

Title Screen

The title screen doesn't do much other than display the title graphic and the “Press Start” text, or any additional text noting if the headset is detached, the Wiimote is not connected, or other error conditions. 

Gameplay Screen

This is where the magic happens.  Perhaps surprisingly, there isn't a ton of code in the GamePlayScreen.cs file itself.  The logic for the game is quite simple:  move the targetting reticule based on the Wiimote IR position, detect if it's over an enemy, if it is, remove hitpoints from the enemy, and if the hitpoints drop below 0, blow up the enemy.  In addition, the enemies fly around the screen and fire missiles at the player.  Let's break down some of these elements.

Enemies

Every drawable object in the game derives from the Drawable2D or Drawable3D base classes.  These classes define some simple properties like position, origin, scale, etc. and provide some methods which must be implemented by the derived classes, like LoadContent, Update and Draw, matching the architecture of a standard XNA application.  Here is what the Drawable2D class looks like:

public abstract class Drawable2D
{
    public Vector2 Position;
    public Vector2 Origin;
    public float Rotation;
    public float Scale;

    public Drawable2D()
    {
        Scale = 1.0f;
        Visible = true;
    }

    public Drawable2D(Vector2 position)
    {
        Position = position;
    }

    // Update/Draw/Load which needs to be implemented by all Drawable types
    public virtual void LoadContent(ContentManager contentManager) {}
    public virtual void Update(GameTime gameTime) {}
    public virtual void Draw(GameTime gameTime) {}

    public bool Visible { get; set; }
}

The Drawable3D class looks almost identical to this, but moves everything to 3D space using Vector3 to denote positions.  It also includes a Model property to hold the geometry of the object, and a property to hold the current World transformation matrix. 

public class Drawable3D
{
    public Vector3 Position;
    public Vector3 Rotation;
    public float Scale;

    protected Model Model;
    protected Matrix World;

    private Vector2 _position2D;

    protected Drawable3D()
    {
        Scale = 1.0f;
        Visible = true;
    }

    public Drawable3D(Vector3 position)
    {
        Position = position;
    }

    public Vector2 GetPosition2D(Camera camera)
    {
        Vector3 position = MindBlasterGame.ScreenManager.GraphicsDevice.Viewport.Project(this.Position, camera.Projection, camera.View, Matrix.Identity);
        _position2D.X = position.X;
        _position2D.Y = position.Y;
        return _position2D;
    }

    // Update/Draw/Load which needs to be implemented by all Drawable types
    public virtual void LoadContent(ContentManager contentManager) {}
    public virtual void Update(GameTime gameTime, Camera camera) {}
    public virtual void Draw(GameTime gameTime, Camera camera) {}
    public bool Visible { get; set; }

The enemy ships and missiles were modeled in 3D Studio Max and exported to the .X file format to be used with the XNA Content Pipeline by kW X-port, a free plugin for 3D Studio Max that can export models and animations to the .X format with a variety of user-controllable options.

Rotate and fly

If you've played the game or watched the video above, you've seen how the enemies rotate toward a position, fly to that position, and then rotate back toward the camera.  To make the ships behave in this way, a random position in 3D space is calculated.  Then the following code is run to rotate the ship to that position:

protected Matrix RotateToTarget(Vector3 position, Vector3 target)
{
    // get direction from position to target
    Vector3 vecToTarget = Vector3.Normalize(target - position);
 
   // get angle to rotate
   double angle = Math.Acos(Vector3.Dot(vecToTarget, Vector3.Backward));

   // get axis to rotate on
   Vector3 axisToRotate = Vector3.Cross(Vector3.Backward, vecToTarget);

   axisToRotate.Normalize();

   // create a rotation matrix from the calculated axis and angle
   return Matrix.CreateFromAxisAngle(axisToRotate, (float)angle);
}

This code uses some vector math to get the direction the random point is in relation to the current ship orientation.  The rotation angle is then calculated, the axis to actually rotate on depending on the location, and finally a rotation matrix is calculated and returned.

This returned rotation matrix is used to rotate the ship over a short period of time using the Vector3.SmoothStep method, which interpolates between two values using a cubic equation.  Once it has reached the target rotation, the ship then “smooth steps” to its destination position, then “smooth steps” back to being rotated toward the camera, using that same RotateToTarget method above, this time with the target position being the camera position.

Using XNA Samples

I used a few samples from the XNA Creators' Club in MindBlaster.  I think there are some fabulous samples available on the site that make life easier for all XNA developers.  I can't speak for everyone, but I'd rather pull out a known working, tested particle engine than spend the time to create one from scratch, especially on a deadline like I had for this project.  Earlier in the article I explained how I used the Game State Management sample to provide the base screen architecture for my application.  I also used its MenuScreen class to build my settings screen as described above.

Particle Engine

I also used the 2D particle engine from the NetRumble sample.  While my game is really in 3D, I was able to use some trickery to throw a particle effect up on the screen in 2D space on top of the ship's location in 3D space which provide a reasonable looking effect.

The particle engine provides sample effects for various explosions and trails.  I used these effects for the enemy ship exploding, the enemy ship firing missiles, and the enemy ship spawning into view.

To get the 2D screen-space position of a 3D object in world-space, I used the following method, which is part of the Drawable3D class:

public Vector2 GetPosition2D(Camera camera)

{
    Vector3 position = MindBlasterGame.ScreenManager.GraphicsDevice.Viewport.Project(this.Position, camera.Projection, camera.View, Matrix.Identity);
    _position2D.X = position.X;
    _position2D.Y = position.Y;
    return _position2D;
}

This uses XNA's Viewport.Project method to return the 2D position of this 3D object using the current 3D position, the current camera projection matrix, and the camera's view matrix and turns it into a 2D point with no additional effort.

So, when an enemy ship explodes or a missile is fired, the 3D object's 2D position is calculated, and the 2D particle effect is displayed at that position.

The background Starfield was also taken from NetRumble.  This is a single class file that can be was easily pulled right out of the sample code and plopped into any XNA application to give a very nice moveable background.

Bloom effect

The Bloom Postprocessor sample on the XNA Creators Club site provides a very easy to use bloom DrawableGameComponent.  A DrawableGameComponent is a class which implements a few methods such as Update and Draw, and then added to the Game object's Component collection.  During each frame, the appropriate methods are called and the object updates and draws.

Blooming is a neat effect that produces some artifacts caused by bright lights.  Think of how your vision reacts when you walk from a dark room out into a bright sunny day.  I use this effect throughout the game to give the ships a neat glowing effect as they heat up.  Also, when an enemy missile hits the player's screen, I flash the screen white for a few milliseconds, and the bloom effect in combination with this produces a nice effect similar to staring at a bright light and then looking quickly looking at something dark as your eyes adjust to the light difference.

Using the BloomComponent is amazingly simple.  In the game's Initialize method:

Bloom = new BloomComponent(this);
Components.Add(Bloom);
Bloom.Settings = BloomSettings.PresetSettings[3];

With this in place, the object does the rest by hooking into the drawing surfaces and applying the effect every frame.  The sample includes several different bloom effects with varying parameters, so its easy to experiment with these values and see what works best for your game.

How to Install and Play

Ensure you have either XNA Game Studio 3.1 installed, or the XNA 3.1 redistributable package.

To play, start the executable and configure the settings that are appropriate to your setup.  For example, you can turn off the Wiimote or the headset if you don't have the hardware to support it.

Up/Down/Left/Right - navigate setup menus
1 - Turn on debug info in game
F1 - Back
F2 - Start
Page Up - Reset the game

If you have turned off Wiimote support in the setup screen, the mouse can be
used to move the targetting reticule.

If you have turned off headset support in the setup screen, the attention
level can be increased/decreased with the left/right mouse buttons.

Move the reticule over an enemy and focus your attention on the enemy to make
your attention score increase.  The higher your attention score, the faster
the enemy heats up and explodes.  As you play, enemies will start to fire back
at you.  Move the reticule over the missiles and focus your attention to blow
them up before they hit you.

Conclusion

There is obviously more to the game than what is seen here, however it would be impossible to cover every single bit of the code in an article here.  I have tried to cover the important and more difficult parts here and show how it's very easy to leverage existing samples to get an XNA game up and running quickly.

If you have any questions or would like further explanation on how a certain portion of the game works, please contact us and I will be happy to answer the questions and perhaps expand the article further.

Thanks

• Joey Buczek for all of the 3d models and artwork in the game
• David Wallimann for all of the sound effects and music in the game
• Rick Barraza for the idea of moving the game into a space environment.  Better than the road I was heading down…
• Greg Hyver and Johnny Liu from Neurosky for getting me set up with headsets and providing loaners for PDC
• Michelle Leavitt for an infinite amount of playtesting
• The XNA folks who created the various tutorials and starter kits whose code saved me a ton of time in creating the game.  Without these, I'd probably still be writing particle engines, bloom shaders and various other pieces from scratch….these are a huge time saver!

Bio

Brian is a Microsoft C# MVP who has been actively developing in .NET since its early betas in 2000, and who has been developing solutions using Microsoft technologies and platforms for even longer. Along with .NET, Brian is particularly skilled in the languages of C, C++ and assembly language for a variety of CPUs. He is also well-versed in a wide variety of technologies including web development, document imaging, GIS, graphics, game development, and hardware interfacing. Brian has a strong background in developing applications for the health-care industry, as well as developing solutions for portable devices, such as tablet PCs and PDAs. Additionally, Brian has co-authored the book "Coding4Fun: 10 .NET Programming Projects for Wiimote, YouTube, World of Warcraft, and More" published O'Reilly. He previously co-authored the book "Debugging ASP.NET" published by New Riders.  Brian is also an author for MSDN's Coding4Fun website.

• PDC09 Seminar:
  Concurrency Fuzzing & Data Races

• Cuzz
• FeatherLite
• Madan Musuvathi
• Sebastian Burckhardt

• Manycore and .NET 4: A Match Made in Visual Studio 2010

And here is the full set of PDC09 sessions related to parallel programming practices:

Overview:

• The State of Parallel Programming

Managed code in Visual Studio 2010:

• Manycore and .NET 4: A Match Made in Visual Studio 2010
• PLINQ: LINQ, but Faster!
• F# for Parallel and Asynchronous Programming

Native code in Visual Studio 2010:

• C++ Forever: Interactive Applications in the Age of Manycore
• Lighting up Windows Server 2008 R2 Using the ConcRT on UMS
• Developing Applications for Scale-Up Servers Running Windows Server 2008 R2

HPC Server:

• Accelerating Applications Using Windows HPC Server 2008

Research and Incubation:

• Axum: A .NET Language for Safe and Scalable Concurrency
• Concurrency Fuzzing & Data Races
• Data-Intensive Computing on Windows HPC Server with DryadLINQ
• Rx: Reactive Extensions for .NET
• Future of Garbage Collection
• Future Directions for C# and Visual Basic

• Sharepoint 2010 Website

• Robby's Blog, Twitter
• Pete's Blog, Twitter
• Tor's Site, Twitter

• Sobees
• Pete's Blog

• Jon's Blog
• Pete's Blog

Learn more about using WPF here.