First Place at SXSW Interactive  [02:16]

Next Steps:
Step #1 – Download the Tools for Windows 8 App Development
Step #2 – Download Visual Studio Express for Windows 8
Step #3 – Start building your own Apps for Windows 8

Subscribe to our podcast via iTunes or RSS

If you're interested in learning more about the products or solutions discussed in this episode, click on any of the below links for free, in-depth information:

Websites:

• Getting started with Windows 8 Apps
• How to Sell Your Apps and Make Money in the Windows Store
• Attend a Windows 8 Developer Camp and Hackathon in your area!

Blogs & Articles:

• Jennifer Marsman's Blog
• Ashley Gullen’s Scirra Blog
• Scirra Competition to build Windows 8 games

Getting Started Tutorials with Construct 2

• Beginners Guide to Construct 2
• Building a Platform Game
• How to Scale to Multiple Form Factors

Videos:

• Microsoft DevRadio: Reasons Why You Should Develop a Windows 8 App
• Microsoft DevRadio: Community Corner - Developing My App for Windows 8

Virtual Labs:

• Windows 8 Release Preview Virtual Labs

The buzz word is REIMAGINATION. The list of new products and technologies launching is endless. Windows 8: Pure Imagination will bring developers, designers, game builders, students and start-ups entrepreneurs together to explore the possibilities of Windows 8. At Windows 8: Pure Imagination you'll be encouraged to look beyond the features... to what can be accomplished when inspiration and technology intersect. Tommy Lee dishes the details about this weekend developer must attend event!

About Windows 8: Pure Imagination

Windows 8: Pure Imagination brings developers, designers, game builders, students and start-ups entrepreneurs together to explore the possibilities of Windows 8. You'll be encouraged to look beyond the features... to what can be accomplished when inspiration and technology intersect. In less than 40 hours, you'll get everything you need to make your vision into the next great Canadian app, including on-site presentations, brainstorming sessions, hands-on labs and access to industry experts for guidance and treated to spontaneous moments of fun to keep your energy up.

Register for Windows 8: Pure Imagination

Register for Windows 8: Pure Imagination - The In-Person Experience (Toronto) >>
Register for Windows 8: Pure Imagination - The Virtual Experience >>

Join the Conversation
Will you be attending Windows 8: Pure Imagination in-person or virtually? What would you want to get out of the weekend? Share your thoughts by starting a new conversation in the Canadian Developer Connection group on LinkedIn. Tommy, Jonathan, as well as other fellow Canadian developers are there networking, sharing, and learning. While visiting the group, make sure to join the group and stay connected.

This segment was presented and recording LIVE on Wednesday, November 5, 2012.

D³: LIVE & INTERACTiVE
In case you haven't heard about the show, Developers, Developers, Developers: LIVE & INTERACTIVE (D³) is a monthly show hosted by Jonathan Rozenblit. The show airs live every first Wednesday of the month at 12:00 PM ET and features the latest updates on what's new and exciting in the world of development; featured presentations; and guests. LIVE and INTERACTIVE means that you'll be part of the show – You're invited to interact with us; ask questions and get them answered; and share your thoughts and opinions.

Join the Canadian Developer Connection LinkedIn group 
Follow @devsdevdevs
Like D³ on Facebook
Subscribe to podcasts via iTunes, Zune, or RSS
 Download the Canadian Developer Connection Windows Store app
Download the Canadian Developer Connection Windows Phone app

More D³: LIVE & INTERACTIVE >>

That's why we invested in DirectX to hardware accelerate almost every aspect of the user experience. Graphics is literally at the core of every Windows 8 experience.

For more information on graphic improvements in Windows 8, see the reference links here:

• Hardware accelerating everything: Windows 8 graphics
• Windows Graphics APIs

See also related videos here:

• Windows 8 for Gaming

We think gamers are going to love the new features of Windows 8.

For additional information on developing games, please see the following links:

• Developing Games
• Creating a DirectX game

See also related videos here:

• Graphic Improvements in Windows 8

Who are you? What do you do at Microsoft?
My name is Salvador Alvarez Patuel, I am a Principal Application Development Manager (ADM). I am responsible for helping customers to develop better solutions using Microsoft technologies. I am currently based in the Microsoft campus in the United Kingdom.

What’s your flight background that made you decide to do this?
I have a private pilot license, obtained 15 years ago, always loved flying and had the opportunity to fly on several different birds, from single piston to multi turbo-prop engines. I have flown several jet planes on real simulators and I try to fly at least once a month on the B737 simulator at British Airways. I have used Microsoft Flight Simulator from the early 90s, always using different hardware devices to enhance my experience, to make it more real.

How did you start this project?
Well, my wife does not like me flying, as she is quite afraid of flying. When we got married she asked me to fly less hours so I tried to negotiate a win-win situation, I said to her “Ok, but can I develop my flight simulator in my studio?”, she obviously answered yes, thinking that I was going to buy some kind of joystick and a bigger monitor . I was aware that there was a small group of people around the world that have built some homemade cockpits in their houses, and started browsing for some ideas. At the beginning it was quite hard, as you can imagine there are not a lot of suppliers that offer affordable parts, so I decided to purchase what I could, and develop what it was missing.

How long did this take you?
I am not considered it done, but I started 3 years ago and started flying just a year ago. That means that it took me 2 years to get some decent parts ready to start experiencing the simulation!

Where did you get all the hardware?
As some parts where available, I try to leverage on affordable products, but sooner realized that I had to create some of the parts. The good thing is that the internet and the community provides most of the diagrams, with measures and materials so it wasn’t very difficult to get what I needed, but it was much harder to make those custom parts using new suppliers. For the structure, I have used Flight Deck Solutions, today they have a very rich set of parts but wasn’t like this when I first started, seems that the hobby is growing!. Some parts are actual real from airplanes, like the yoke which I rewire, add some potentiometers connected to a USB board to get the values from it. My latest addition, the throttles where custom made by a third party company called Revolution Sim Products as with my new baby boy I didn’t have time to build it myself.

How did you get it all to work with FSX?
That was the interesting part, as I am a software developer I had all the necessary skills. I started developing all the software by myself, reading real Boeing 737 manuals with more technical drawings than the space shuttle (trust me!) I coded the aircraft logic as I was just using the 737 models in flight simulator. I knew about FSUIPC library, written by Peter Dawson and I start interfacing my software with FSX using the available offsets. For the visual avionics I wrote components using DirectX, but rapidly found myself diverting a lot of energy getting the visual rights, and not enough time on the logic. Therefore, I decided to buy a visual package from Sim Avionics, at that time it was a very cheap option compared with Project Magenta, one of the leaders on this area, now they are similar prices. I use my system logic to interact with Sim-Avionics using FSUIPC offsets, this is a very fast input/output model that allows me to link 3 computers in near real time. The other hard part was to write components that read USB values, from all my hardware. I have managed to connect everything together but trust me, it was quite hard

What do you use for your primary displays?
My first idea was to use a set of projectors, I bought one but I found that without a proper cockpit shell it didn’t look real, just a big image on the wall. So I decided to take a different approach, using several monitors. The problem was that a decent graphic accelerator card will have two outputs, and the performance degrades when you increase the resolution and detail, so I found another way to do it. I found the Matrox triple head device as an excellent alternative, as it takes the feed from one output and spread it across 3 screens, without degrading the performance, now it looks like I have proper cockpit windows!

As you may know, FSX is hungry for resources so I must admit that I have invested on the main machine that only runs FSX. It has an i7 processor, 8Gb DDR3 RAM , solid state drives and an nvidia GTX 570 graphic card. I am getting 40fps nearly full detail, sweet.

How much of the controls actually work compared to just being “eye candy”? 
Practically everything I have works as the real plane, even the lights dimmer. The motored throttles have given it the final touch, as follows the autopilot logic that I have also coded. I have some dummy parts still, but just because I didn’t have the time, like the chronometer and the yaw dumper gauge. The other area where software is replacing hardware is the overhead panel and the pedestal area (with radio controls), I don’t have the space to do it but once I have more space I will definitely jump into them.

What do you have left that you’d like to add?
I want to add another computer to manage the CDU (as is quite intensive) and the lower display unit, just beside it. It will require a custom LCD screen (as the upper display unit that I already have) and some wiring. That will allow me to display secondary information, like tire pressure, doors, etc. Also I want to add the first officer side, not that I needed but it will look nicer.

Do you think you could fly a real 737 now after having this?  
I already fly the real 737 simulators, therefore my answer will be “Probably yes”.

About Nickolas Landry

About Rob Cameron

Nickolas Landry and Rob Cameron recommend you check out

• Download the Windows Phone 7 developer tools
• Register for Marketplace and upload your app
• App Hub Windows Phone 7 Developer Portal
• Learn more about Windows Phone 7 development with ""How Do I"" videos
• Windows Phone 7 Developer Training Kit
• Windows Phone 7 Jump Start video training
• Silverlight
• Microsoft Silverlight 4 Download
• Nick's blog
• Infusion

West coast fans won’t be missing out though – at the Music Project | Science Fiction Museum and Hall of Fame in Seattle, Bungie developers will greet fans on Sept. 13th during the countdown to launch. There will also be prizes and fans will get to play the game early there, too.

Even if you’re not located in one of these cities, a number of retailers will be holding their own celebrations, including Best Buy, GameStop, Wal-Mart, Toys”R”Us and many others, so check out your own local shops for details.

The game is still in development, but an alpha version is almost ready for internal testing, Kevin Unangst, a senior director in Microsoft's game unit, recently told CNET News. But don’t expect a public beta this year, he said.

Details on gameplay are few, as are screenshots, but the “Flight” website does have a  teaser video available which shows a prop plane soaring over sparkling water as the sun rises, casting everything with a an orange-ish glow.

“Flight” isn’t the only game to be resurrected. Another classic, “Age of Empires,” is also returning in a new online version and it is taking sign ups for its beta launching in 2010. According to the news release, Age of Empires invites players to watch their own Greek civilization grow from village to empire, while embarking on quests and adventures, either solo or with friends.

You can learn more about all these games and other new releases at microsoft.com/games.

The traditional analog elements of the game are now touchable, interactive graphics appearing on the Surface computer’s screen. And the game also hooks up to the Web, allowing for gamer accounts, leader boards and more. Users can even play each other over the Internet.

To see what Catan looks like on Surface, check out this photo gallery or this YouTube video. It definitely looks like fun!

But don’t worry if you’re not sure what game to search for by name – I discovered that you can simply type in “games” into Bing and the top result will link you directly to the Games themselves (click either the link’s heading or just “Play Now.”) Here you’ll find all the games housed on Bing, like the ever-popular Bejeweled, for example, and Plants vs. Zombies, just to name a couple.

The bing.com/games site also groups games into categories like “most popular,” “action favorites,” “puzzle favorites,” “word favorites,” and more, so you can find just the sort of game you’re looking for, even  if you don’t know what it’s called.

In the music section, you can find lyrics, photos, videos and tour dates for your favorite artist or band, but even more notable, you can stream music. By integrating Zune into Bing, there’s access to over 5 million songs available for your listening pleasure. And if you want to buy a track, Bing points you to Zune, iTunes and Amazon.com’s MP3 store.

A second integration from Microsoft Games provides access to detailed information on over 35,000 games including reviews, cheats, walkthroughs and more. Plus, almost 100 casual games are available to play right from within Bing. Just search for a game and then click the “Play Now” button that appears under the listing.

Finally, there’s video. Bing now offers thousands of episodes from 1500+ shows, thanks to new partnerships with Hulu, Viacom and CBS. This “Bing TV” section will serve as a one-stop-shop for online TV viewing, offering everything from clips to full episodes and even local listings.

Movies aren’t left out either: reviews from critics, Facebook and Twitter sentiment, and other info from Bing Maps that helps you navigate to your favorite theater are included here.

Everything except the TV listings should be live now – those will come over the next couple of weeks. Zune music playback is also ramping up now – more songs will be available in a few days.

Multi-Targeting

Windows Phone 7 uses a version of Silverlight 3 with a few bonus features. Because this version of Silverlight is so close to the Web version of Silverlight 3, we'll take the approach of Multitargeting a solution for both platforms. To do so, start with a Silverlight 3 (web) application template, and then add in a Windows Phone project with linked files that re-use the Silverlight 3 solution. This allows us to deploy to many different platforms using the same code base.

Let's get started by creating our Solution in this way.

Creating the Solution and MainPage

1. In Expression Blend 4, create a new Silverlight Application + Website template named Shuffleboard. Be sure to select “3.0” from the Version dropdown, as this is what is supported by Windows Phone (in a little bit we'll add in the Windows Phone project template as well).
   
2. In the Objects and Timeline Panel, select the main UserControl and set its Width to 480 and its Height to 800. This is the size of a Windows Phone UI in Portrait mode.
3. Select the LayoutRoot Grid and set its Width to 480 and its Height to 800.
4. For games, a Canvas layout container is better than a Grid, so let's change the LayoutRoot container type. Right-click LayoutRoot and select Change Layout Type/Canvas.
   
5. Since some parts of our UI can appear outside of the game area, we want to add a Clip to this main canvas so that the user does not see these outside elements. Add the following just under the LayoutRoot Canvas:
   This XAML is snippet “MainPage Clip” inside snippets.txt.

   <Canvas.Clip>
       <RectangleGeometry Rect="0,0,480,800"/>
   </Canvas.Clip>

6. We'll be using some pre-built Behaviors that make it easy to introduce physics into Silverlight using the Farseer Physics Library. Right-click the Silverlight project and select Add Reference. Select the following assemblies, located in the sample download:
   
   \ShuffleBoard\Bin\Debug\FarseerPhysics.dll
   \ShuffleBoard\Bin\Debug\Spritehand.FarseerHelper.dll
   \ShuffleBoard\Bin\Debug\Spritehand.PhysicsBehaviors.dll
   
   NOTE: To learn more about the Physics Helper Library, visit http://physicshelper.codeplex.com

Creating the Game User Control

Next we'll create a separate user control that will contain the core game logic. By creating a separate user control, we can easily target different platforms by placing the user control inside different MainPage elements.

1. Right-click the Shuffleboard Silverlight project and select Add New Item. Select UserControl and name the control “ucMainGame”.
   
2. Change the LayoutRoot element from a Grid to a Canvas.
3. Set the Width to 480 and the Height to 800 for both the UserControl and LayoutRoot Canvas elements.
4. Set the Background color to Black for the LayoutRoot Canvas.
5. Build the project by pressing Ctrl+Shift+B.
6. Back on MainPage, add an instance of ucMainGame to the page by going to the Assets Panel and expanding Controls/All. Drag an instance of ucMainGame to the artboard.
   
7. Rename the ucMainGame control to ucMainGame1.
8. Set the Left and Top properties of ucMainGame1 to 0.

Creating the Table

We'll use a pre-rendered PNG image for the shuffleboard table.

1. Create a new subfolder in the Silverlight project named “images”.
2. Add the following JPG image into the new images subfolder, located in the sample download:
   \ShuffleBoard\images\shuffleboardTable.jpg
3. Open the ucMainGame user control.
4. Insert a new Canvas element named cnvTable, and set the following attributes:
   1. Width = 480
   2. Height = 800
   3. Left = 0
   4. Top = 0
5. In the Assets Panel, expand the Behaviors category and drag a PhysicsControllerBehavior on the cnvTable. This behavior introduces the core physics simulation into the game. Set the properties of the Physics Contoller as follows (this turns of gravity and sets some other physics parms):
   
6. Inside the cnvTable Canvas, add a second Canvas named cnvTableInner, which will hold the table image. Set the following properties on cnvTableInner:
   1. Width = 360
   2. Height = 1077
   3. Left = 60
   4. Top = -277
7. Drag the shuffleboardTable.jpg image from the Projects Panel into cnvTableInner. Set the following properties on the image:
   1. Width = 360
   2. Height = 1077
   3. Left = 0
   4. Top = 0
8. The Objects and Timeline Panel should look like so:
   
9. We will need to detect when a puck hits the edge of the table and “falls off.” Let's add in some Rectangle elements and add Physics Behaviors to them.
10. Add a new Rectangle named rectSensorLeft and size it so that it covers the entire left side of the table. Set the following properties:
    1. Width = 60
    2. Height = 1190
    3. Left = -40
    4. Top = -334
    5. Opacity = 20%
11. Drag a PhysicsObjectBehavior from the Assets Panel Behaviors onto rectSensorLeft, and then set its IsStatic property to true.
12. Add three more Rectangles by copying rectSensorLeft, so that they surround the borders of the table:
    1. rectSensorRight
    2. rectSensorTop
    3. rectSensorBottom
13. Your artboard should look similar to the following:
    
14. Now we'll add in the goal Rectangles at the end of the table. These will be used to determine the number of points a puck receives. Draw out three Rectangles that cover the width of the board and are 100 pixels high. Name these rectPoints3, rectPoints2, and rectPoints1. Set their Stroke to Red and their Fill to No brush.
15. Add a TextBlock element inside each Rectangle to depict the points. Set the text properties to 3, 2, and 1 so that the artboard looks similar to the following:
    
16. We'll also need a Rectangle representing the “slide zone” where players can legally slide a puck (we don't want to allow sliding all of the way down the table!). Add a Rectangle named rectInBounds and position it at the lower end of the table:
    1. Width=360
    2. Height=292
    3. Left = 60
    4. Top = 508
    5. Fill = No Brush
    6. Stroke = Red

Adding the Pucks

For the pucks, we are going to add in an existing control which has the artwork completed.

1. Right-click the project and select Add Existing Item.
2. Browse to the following two files in the sample code download:
   \ShuffleBoard\ucPuck.xaml
   \ShuffleBoard\ucPuck.xaml.cs
3. Open ucPuck.xaml on the artboard and notice the following in the Objects and Timeline Panel:
   
   1. There is a Canvas named Puck that has a PhysicsObjectBehavior applied to it. This allows each instance of this Canvas to behave like a Physics Object so it animates with velocity and force and participates in collisions. Note that this Behavior has a large MomentOfInertia value. This keeps the object from rotating due to torque and collisions. Also note the RestitutionCoefficient, which gives the object some “bounce.”
      
   2. There is a “cnvInner” Canvas that defines the look of the Puck.
   3. There is a StoryBoard defined, sbLostPuck, which we will execute when the puck “falls off the edge” of the table.
   4. Open the code-behind file, ucPuck.xaml.cs, and note the get / set property for the color of the puck. This will allow us to create both Red and Blue pucks for each player.
4. Build the project by hitting Ctrl+Shift+F5 and go back to ucMainGame.
5. From the Assets Panel, expand Controls/All and find the ucPuck control. Drag an instance of this onto the artboard. Name this bluePuck1.
6. Copy and paste bluePuck1 twice so that you have three blue pucks. Name the new pucks bluePuck2 and bluePuck3.
7. Copy and paste a fourth puck and name it redPuck1. In the Properties Panel, go to the Miscellaneous Category and change the ColorHighlight and ColorMain properties to a Red color:
   
8. Copy and Paste redPuck1 two times and rename these new pucks redPuck2 and redPuck3.
9. Let's test our table. Find the PhysicsControllerBehavior just under the cnvTable Canvas in the Objects and Timeline Panel and set the MousePickEnabled property to true.
10. Run the project by clicking F5. Try dragging some pucks around with the mouse.

Adding the Scoreboard

We need a way of tracking players' scores, so let's add in a simple scoreboard for Blue vs. Red.

1. Right-click the project and select Add Existing Item.
2. Browse to the following two files in the download:
   \ShuffleBoard\ShuffleBoard\ucScoreBoard.xaml
   \ShuffleBoard\ShuffleBoard\ucScoreBoard.xaml.cs
3. Open ucScoreBoard and notice the following:
   1. It has a TextBlock for Red scores and a TextBlock for Blue scores.
   2. In the code-behind, there are setters and getters to set the score.
4. Build the project by selecting Ctrl+Shift+B.
5. Switch back to ucMainGame.
6. Drag an instance of ucScoreBoard from the Assets Panel onto into LayoutRoot.
7. Name the element ucScoreBoard1, and position it at the top left of the Canvas like so:

Adding a Player Up Display

We'll need a simple control that displays which player's turn it is.

1. Right-click the Silverlight project and select Add/Existing Item.
2. Browse to and select the following two files in the sample folder:
   \ShuffleBoard\ucPlayerUp.xaml
   \ShuffleBoard\ucPlayerUp.xaml.cs
3. Note the following about ucPlayerUp:
   1. The code-behind file has a simple property, IsBlueTurn, which shows the appropriate message in the UI.
4. Build the project by clicking Ctrl+Shift+B
5. Back on ucMainGame, drag an instance of ucPlayerUp into cnvTableInner.
6. Name the control ucPlayerUp1 , and set the following properties:
   

Adding a 3D Look

Let's give our table a bit of a 3D look by adding a Perspective transform so the table appears to go off into the distance.

1. Create a new StoryBoard named sbRotateTable.
   
2. Select the cnvTable Canvas element.
3. Advance the timeline ahead one second.
4. In the Properties Panel, under Projection, set the X value to -40.
   
5. Your artboard should now look something like this:
   
6. Close the Storyboard to end recording.
   
7. We want to rotate the table after the PhysicsController is initialized so the Physics Helper can determine the boundaries of the physics objects properly. Start by adding in some Using statements at the top of ucMainGame.xaml.cs:
   This code is snippet “ucMainGame Imports” inside snippets.txt.

   using System.Collections.Generic;
   using Spritehand.FarseerHelper;
   using FarseerGames.FarseerPhysics.Mathematics;
   using System.ComponentModel;

8. Next, let's add in some class-level declarations, which will store the physics controller, list of pucks, and some variables used to track shooting and scoring:
   This code is snippet “ucMainGame Declarations” inside snippets.txt.

   PhysicsControllerMain _physicsController;
   List<PhysicsSprite> _pucks = new List<PhysicsSprite>();
   List<PhysicsSprite> _redPucks = new List<PhysicsSprite>();
   Point _ptStartShot;
   bool _draggingPuck;
   int _draggingStartTick;
   
   int _currentPuckToShoot = 0;
   int _currentPuckShot = -1;
   int _gameOverScore = 15;

9. In the ucMainGame() constructor, add some code to set the MaxFrameRate to 30 (Windows Phone will be limited to 30 FPS) and wire up the Loaded event handler.
   This code is snippet “ucMainGame Constructor” inside snippets.txt.

   this.Loaded += new RoutedEventHandler(ucMainGame_Loaded);
   Application.Current.Host.Settings.MaxFrameRate = 30;

10. Implement the ucMainGame_Loaded event handler, which gets a reference to the Physics Controller and wires up our event handlers.
    This code is snippet “ucMainGame Loaded” inside snippets.txt.

    void ucMainGame_Loaded(object sender, RoutedEventArgs e)
    {
        if (DesignerProperties.GetIsInDesignMode(this))
            return;
    
        _physicsController = 
            cnvTable.GetValue(
                PhysicsControllerMain.PhysicsControllerProperty
            ) as PhysicsControllerMain;
        _physicsController.Initialized += _physicsController_Initialized;
        _physicsController.Collision += _physicsController_Collision;
        _physicsController.TimerLoop += _physicsController_TimerLoop;
    }

11. Next, we'll add in the event handlers that we just wired up in the loaded event. Note that the Initialized event starts the Rotate Table StoryBoard.
    This code is “ucMainGame Event Handlers” inside snippets.txt.

    void _physicsController_Initialized(object source)
    {
        sbRotateTable.Begin();
    }
    
    void _physicsController_Collision(string sprite1, string sprite2)
    { }
    
    void _physicsController_TimerLoop(object source)
    { }

12. Run the Project by clicking F5. Note the 3D look and try manipulating the pucks with the mouse.

Controlling the Pucks

Next, we'll add logic to control the player's turn and control and along with score. Note that, when we implement the Windows Phone version in a bit, we can take advantage of Multitouch Manipulation events. But since these events are not available in the Web version of Silverlight 3, we'll use a simple mouse input mechanism.

1. Let's turn off the default mouse manipulation. Select the PhysicsControllerBehavior just under the cnvTable Canvas and set the MousePickEnabled property to false.
2. We need to get references to our Pucks, which the Physics Helper has translated into PhysicsSprite objects. A Physics Sprite contains the XAML UI for the Physics Object, plus the Physics Engine attributes for the underlying physics objects–including boundary shape, mass, velocity, etc.
   This code is “ucMainGame Initialized” inside snippets.txt.
3. Now we need to handle the Mouse events on the pucks in order to control the player's shot. When the player clicks the mouse down on a puck, we track the position along with time that they clicked. When the player moves the mouse, we update the puck position and also check if the player paused or moved backwards. We do this in case the player is just adjusting the puck position and not yet sliding the puck for a shot. Finally, in the Mouse up event, we release the puck in the specified direction and apply an appropriate amount of force to it.  Copy in the snippet “Puck Mouse Control” from snippets.txt.  Code is not shown here for brevity.
4. When the Physics Helper Library does its magic, it translates existing UI Elements into PhysicsSprite objects. So what if we want to get at the original controls, perhaps to execute a StoryBoard that we defined on them? We can do this by using the FindName method to get the original user control instance. So, add the code below, which we'll use to get a reference to the Puck Storyboard “sbLostPuck”.

   ucPuck GetPuckControl(string spriteName)
   {
       ucPuck puck;
       var parent = PhysicsControllerMain.ParentCanvas;
       switch (spriteName)
       {
           case "Puck":
               puck = parent.FindName("bluePuck1") 
                       as ucPuck;
               break;
           case "Puck_1":
               puck = parent.FindName("bluePuck2") 
                       as ucPuck;
               break;
           case "Puck_2":
               puck = parent.FindName("bluePuck3") 
                       as ucPuck;
               break;
           case "Puck_3":
               puck = parent.FindName("redPuck1") 
                       as ucPuck;
               break;
           case "Puck_4":
               puck = parent.FindName("redPuck2") 
                       as ucPuck;
               break;
           default:
               puck = parent.FindName("redPuck3") 
                       as ucPuck;
               break;
       }
   
       return puck;
   }

5. Finally, to score, we'll determine if the puck lies between any of the Rectangles in the end zone.
   This code is the snippet “Get Points for Puck” in snippets.txt.

   int GetPointsForPuck(PhysicsSprite puck)
   {
       int score = 0;
       Vector2 puckPos = puck.BodyObject.Position;
   
       double left = Convert.ToDouble(
           rectPoints3.GetValue(Canvas.LeftProperty));
       double top = Convert.ToDouble(
           rectPoints3.GetValue(Canvas.TopProperty));
   
       if ((puckPos.X > left && puckPos.X < left + rectPoints3.Width) 
        && (puckPos.Y > top && puckPos.Y < top + rectPoints3.Height))
           score = 3;
   
       left = Convert.ToDouble(rectPoints2.GetValue(Canvas.LeftProperty));
       top = Convert.ToDouble(rectPoints2.GetValue(Canvas.TopProperty));
       if ((puckPos.X > left && puckPos.X < left + rectPoints2.Width) 
        && (puckPos.Y > top && puckPos.Y < top + rectPoints2.Height))
           score = 2;
   
       left = Convert.ToDouble(rectPoints1.GetValue(Canvas.LeftProperty));
       top = Convert.ToDouble(rectPoints1.GetValue(Canvas.TopProperty));
       if ((puckPos.X > left && puckPos.X < left + rectPoints1.Width) 
        && (puckPos.Y > top && puckPos.Y < top + rectPoints1.Height))
           score = 1;
   
       return score;
   }

Implement the “Game Loop”

Most games are controlled by a “Game Loop” that executes many times per second. Within this Game Loop, we can check for collisions, perform enemy AI, and scoring. The PhysicsController fires an event called “TimerLoop” that we can use for this purpose.

1. Replace the existing TimerLoop and Collision events with the following code snippet, which checks to see if any pucks have been shot and if we are ready for the next shot. Do this by seeing if the puck velocity has slowed down to nearly a stop…
   This code is snippet “Timer Loop” inside snippets.txt.

   void _physicsController_TimerLoop(object source)
   {
       // check to see if the current shot is completed
       if (_currentPuckShot >= 0)
       {
       var puck = _pucks[_currentPuckShot].BodyObject;
   
           if (puck.Enabled == false ||
             Math.Abs(puck.LinearVelocity.X) < 3                                   
             && 
             Math.Abs(puck.LinearVelocity.Y) < 3)
           {
               // did the shot clear the end zone?
               if (!PointWithinBounds(
                 new Point(puck.Position.X, 
                          puck.Position.Y)))
               {
                   _currentPuckShot = -1;
                   _currentPuckToShoot++;
                   SetupThePuck();
               }
           }
       }
   }
   
   void _physicsController_Collision(string sprite1, string sprite2)
   {
       // check for puck off sides
       if (sprite1.StartsWith("rectSensor") && sprite2.StartsWith("Puck"))
       {
           ucPuck puck = GetPuckControl(sprite2);
   
           _physicsController.PhysicsObjects[sprite2].
               BodyObject.Enabled = false;
   
           puck.sbLostPuck.Begin();
       }
   }

Targeting Windows Phone

So far, we've created a Silverlight 3, web-based version of a shuffleboard game. Next we'll quickly port this to Windows Phone, taking advantage of some of the platform's capabilities, such as multitouch. We'll do this by using linked files that point back to our existing Silverlight 3 project.

1. Right-click the Silverlight solution in Blend and select Add New Project.
2. Select Windows Phone Application and enter ShuffleBoard.WindowsPhone for the name.
   
3. Delete the TitleGrid and ContentGrid elements from the Objects and Timeline Panel.
4. Convert LayoutRoot to a Canvas by right-clicking it and selecting Change Layout Type/Canvas.
5. Right-click the project and select Add Reference. Browse to the following assemblies located in the sample download (these are the WindowsPhone versions of the Physics Helper and Farseer):
   
   \ShuffleBoard.WindowsPhone\Bin\Debug\FarseerPhysics.dll
   \ShuffleBoard.WindowsPhone\Bin\Debug\Spritehand.FarseerHelper.dll
   \ShuffleBoard.WindowsPhone\Bin\Debug\Spritehand.PhysicsBehaviors.dll
   
6. We need references to the assemblies used for Behaviors. An easy way to do this is to add a Behavior to an element and then delete it. From the Assets panel, drag a PhysicsControllerBehavior onto LayoutRoot and then delete it. Note that this adds a reference to System.Windows.Interactivity to the project.
7. Next, we'll add in the linked files from the existing Silverlight 3 project. Right-click the ShuffleBoard.WindowsPhone project and select Add/Link to Existing Item. Select the following files:
   1. ucMainGame.xaml
   2. ucMainGame.xaml.cs
   3. ucPlayerUp.xaml
   4. ucPlayerUp.xaml.cs
   5. ucPuck.xaml
   6. ucPuck.xaml.cs
   7. ucScoreBoard.xaml
   8. ucScoreBoard.xaml.cs
8. 8. Add a new folder to the project named images.
9. 9. Right-click the images folder and select Link to Existing item, then navigate to the following image in the sample folder:
   \ShuffleBoard\images\shuffleboardTable.jpg
   
10. Build the project by clicking Ctrl+Shift+B.
11. From the Assets Panel, under Controls/All, find ucMainGame and drag an instance onto LayoutRoot. Set the Left and Top properties to 0.
12. Right-click the WindowsPhone project and select “Startup” to set this as the startup project.
13. Run the project by clicking F5.

Windows Phone Touch Input

So far, we've used a simple mouse-event-based input mechanism for shooting the pucks. But on the Windows Phone 7 platform, we can do better than that by using Multitouch events. These events include an inertia property that will make our puck physics more realistic.

1. Since we are going to need to support two different platforms now (Web and Windows Phone), we need to introduce a Conditional Compilation Symbol so that the compiler can differentiate between each platform's code.
2. Open the solution inside Visual Studio by right-clicking it in the Projects Panel and then select “Edit in Visual Studio”
   
3. Right-click the ShuffleBoard.WindowsPhone Project and select Properties.
4. 4. Add in a new compilation symbol for WINDOWS_PHONE.
   NOTE: Future releases of WP7 Tools will likely define a default compilation symbol, so you may already see one defined here.
   
5. Now we'll add in the Windows Phone-specific events for manipulation using touch. Open ucMainGame.xaml.cs and find the _physicsController_Initialized event handler. Replace the events for the mouse events with the following, which will wire up either the manipulation events (for Windows Phone) or the mouse events (for Silverlight Web):
   This code is snippet “Manipulation Events” inside snippets.txt.

   #if WINDOWS_PHONE
       puck.ManipulationStarted += 
         new EventHandler<ManipulationStartedEventArgs>
         (puck_ManipulationStarted);
       puck.ManipulationCompleted += 
         new EventHandler<ManipulationCompletedEventArgs>
         (puck_ManipulationCompleted);
       puck.ManipulationDelta += 
         new EventHandler<ManipulationDeltaEventArgs>
         (puck_ManipulationDelta);
   #else
       puck.MouseLeftButtonDown += 
         new MouseButtonEventHandler(puck_MouseLeftButtonDown);
       puck.MouseMove += 
         new MouseEventHandler(puck_MouseMove);
       puck.MouseLeftButtonUp += 
         new MouseButtonEventHandler(puck_MouseLeftButtonUp);
   #endif

6. Now we just need to implement those event handlers for manipulation. The main consideration here is that the ManipulationCompleted event passes in a FinalVelocity.LinearVelocity parameter, which we can use to create realistic physics from the user's shot:

   Add the code snippet “Manipulation Event Handlers” from snippets.txt

   Vector2 force = new Vector2(
       (float)(e.FinalVelocities.LinearVelocity.X * scalePower), 
       (float)(e.FinalVelocities.LinearVelocity.Y * scalePower));

7. Run the project by clicking F5, and try sliding some pucks.

Performance

Depending on your configuration, you may experience poor performance for the shuffleboard game when running it in the Windows Phone emulator. There are different reasons for this, including video card drivers and virtualization settings. Please refer to this blog post, which details these performance considerations.

1. 1. Inside the ucMainGame constructor, let's turn on the Frame Rate counter so we can see our current frame rate. This code is snippet “Frame Rate Counter” inside snippets.txt

   Application.Current.Host.Settings.EnableFrameRateCounter = true;

2. Run the project by hitting F5, and note the current frame rate.
   NOTE: Your development configuration may prevent you from having an adequate framerate in the Windows Phone Emulator! Please refer to this blog post for tips on emulator performance.

One performance tweak we can easily make is taking advantage of hardware acceleration. Silverlight enables us to use the video card to render elements, which can greatly increase performance. To do this, we add a Cachemode=”BitmapCache” attribute to any elements we want to hardware accelerate.

1. Open ucPuck.xaml and note that there is a CacheMode attribute on the Puck Canvas.
2. Open ucMainGame.xaml and add a CacheMode attribute to the cnvTable Canvas element.

   <Canvas x:Name="cnvTable" Width="480" Height="800" 
   d:LayoutOverrides="Width, Height" CacheMode="BitmapCache">

3. Run the project and note the frame rate change.

Adding Sound

Sound is one area where Windows Phone and Web Silverlight differ a bit. Under Silverlight for Web, we can use the MediaElement class to play back WMA and MP3 format audio. Multiple instances of the MediaElement class can be played simultaneously, and the sound output will be automatically mixed.

However, on Windows Phone we have access to the XNA game libraries, including sound support. This is a much more efficient way to add mixed sound to a Silverlight game on Windows Phone, but it only supports WAV format sound files.

The Physics Helper Library has a useful class wrapper for sounds, which we'll use to play a “hit puck” sound on both platforms.

1. In the ShuffleBoard (Silverlight Web) project, right-click and select Add Folder.
2. Name the folder sounds.
3. Add an existing item to the sounds folder from this location:
   \ShuffleBoard\ShuffleBoard\sounds\hitPuck.wma
4. Set the Build Action of the file to Content.
5. In the ShuffleBoard.WindowsPhone project, right-click and select Add Folder.
6. Name the folder sounds.
7. Add an existing item to the sounds folder from this location:
   \ShuffleBoard.WindowsPhone\sounds\hitPuck.wav
8. Set the Build Action of the file to Content.
9. Inside ucMainGame.xaml.cs, add a class-level declaration for a SoundMain class (this is defined in the Physics Helper Library):
   This snippet is “Declare the Sound” inside snippets.txt.

   SoundMain _soundPuckHit;

10. 10. Inside the ucMainGame_Loaded event, add code to declare the two sounds. Note that Windows Phone uses a WAV format sound without a “/” prefix for the root.
    This snippet is “Declare the Sound” inside snippets.txt.

    #if WINDOWS_PHONE
        _soundPuckHit = new SoundMain(this.LayoutRoot,
                       "sounds/hitPuck.wav", 2, 0);
    #else
        _soundPuckHit = new SoundMain(this.LayoutRoot, 
                       "/sounds/hitPuck.wma", 2, 0);
    #endif

11. Inside the _physicscontroller_collision event, add code to play the sound if a puck-to-puck collision occurs and the velocity is > 10:
    This snippet is “Play the Sound” inside snippets.txt.

    // check for puck to puck collsion
    if (sprite1.StartsWith("Puck") && sprite2.StartsWith("Puck"))
    {
        PhysicsSprite sprite = _physicsController.PhysicsObjects[sprite1];
    
        if (Math.Abs(sprite.BodyObject.LinearVelocity.X) > 10
            || Math.Abs(sprite.BodyObject.LinearVelocity.Y) > 10)
        {
            _soundPuckHit.Play();
        }
    }

12. Run the project and try hitting pucks together.

Summary

Since the 1.1 Alpha release, Silverlight has offered a compelling casual game development environment. With the addition of Windows Phone, along with demos of Silverlight running on other embedded devices, you can bet the future of Silverlight for gaming is bright.

Here are a few resources for more information:

Andy's blog
http://www.andybeaulieu.com

Andy's Silverlight Demos
http://www.spritehand.com

The Physics Helper Library
http://physicshelper.codeplex.com

Great free resource for learning Expression Blend and Silverlight
http://www.microsoft.com/design/toolbox/

MTV properties will also cover Xbox 360 game announcements, world premieres and other breaking news throughout the conference. 

But wait, it gets even better: Cirque du Soleil will be there too, turning E3’s media circus into a real one. Apparently, the Cirque du Soleil performers will be helping to introduce Natal, which captures body motions via sensors and a camera and translates those into in-game actions. That should be interesting!

On Monday June 14th, Spike TV will cover E3 media briefings and game launches live at 10:30 AM PDT. On Tuesday, June 15th, MTV will cover Natal’s launch starting at 12:30 PM PDT. That coverage will be rebroadcast on Nick at Nite, mtvU, MTV Hits and Logo at 6 PM PDT later that day.  Also, MTV Networks’ GameTrailers.com will provide live, high-definition video streams of both these events and more.

Or if you must be there in person, E3 will be held at the Los Angeles Convention Center from June 13th to 18th.

Introduction

This is an advanced tutorial, and as such we expect that you are already familiar with XNA and scene graph based 3D programming. The game is a single player game in which the player shoots virtual balls into a real-world environment to knock down all of the virtual dominos overlaid on top of a board. The player holds a webcam in one hand and shoots the virtual ball by clicking a mouse using the other hand. Before the game starts, the player is allowed to add virtual dominos or modify the positions and orientations of the existing ones overlaid on the board as desired.

How To Compile and Run The Game

After you download the Goblin XNA distribution from http://goblinxna.codeplex.com, as well as all other necessary libraries indicated in the installation guide, compile the Goblin XNA project under the /src directory. If you're not familiar with Goblin XNA framework, it is strongly recommended that you first go through the tutorials in the /tutorials directory. Then, find the ARDominos directory under /projects directory and open up the solution file in order to build and run the game.

How To Play

Before you run the application, you need to print out the marker array. The marker array can be found in the DominoGroud.ppt file. The 1^st slide is for the ARTag library, and the 2^nd one is for the ALVAR library. The size of the PowerPoint slide is set to 25.6 x 17.52 inches, so if you want the entire marker array to fit in A4 size paper, you should choose "Scale to fit" option when you print. Once you have the print out, you're ready to run the game.

The virtual dominos are overlaid around the center of the marker array (referred to as ‘game board' from now on), so you need to aim your webcam around the center of the game board. If you move your mouse cursor around on top of the game window, you will see a yellow-green cross-hair cursor, and this cursor position will be used to control the game play. All of the mouse-based interactions are handled by the left mouse button.

The initial game mode is ‘Add' (as indicated on the upper-right corner of the game window), so if you click on any part on the game board, a new domino will be added. If you press-and-hold, then you can move the newly added domino around, and once you release the mouse the domino will be added at the released position. While the domino is rendered in transparent, you are allowed to modify its orientation along the normal vector of the game board by using the ‘Left' and ‘Right' arrow keys. You are not, however, allowed to add a domino outside of the gameboard, and you are initially limited to maximum of 40 dominos on the game board (you can change this limit in the code if you want).

If you want to edit the current position or orientation of an existing domino on the game board, change the game mode to ‘Edit' by clicking the ‘E' key. To modify the position, press-and-hold the mouse on the desired domino, and drag it around (again, it won't allow you to place the domino outside of the game board). The selected domino is rendered transparent, and it remains selected until you select another domino or change to a different game mode. If a domino is selected, you can change its orientation by using the ‘Left' and ‘Right' arrow keys. If you want to delete a domino, select the domino you want to delete and press the ‘D' or ‘Delete' key.

Once you are satisfied with the current domino configuration, start the game by pressing the ‘P' key. If you want to reset the configuration to the original one, then press the ‘R' key. Once the game starts, the elapsed time will display in the upper-right corner and the number of balls shot will display in the lower-right corner. The goal of the game is to knock all of the virtual dominos off the game board by shooting as many virtual balls as necessary, and you will get different trophies based on the elapsed time.

The virtual ball is shot from the cursor location when you click the mouse. You can move the webcam around to move closer to certain dominos and improve your chances of hitting your target. However, if the game board becomes invisible to the camera, the game doesn't allow you to shoot any balls.

Once you win, you need to press the ‘P' key to restart the game.

List of command keys used in the game:

· 'H' -- Display short-cut key help menu

· 'A' -- Switch to "Add" mode

· 'E' -- Switch to "Edit" mode

· 'P' -- Switch to "Play" mode. If pressed during "Play" mode, it restarts the game.

· 'R' -- Reset the game to the initial state

· 'S' -- Toggles the shadow mapping (NOTE: On non-NVidia graphics cards, the shadow mapping may have some aliasing problem)

· 'G' -- Toggles the GUI

· 'D' -- Delete selected dominos during "Edit" mode

· 'C' -- Toggle center cursor mode during "Play" mode

Step 1. Setup Scene Basics

First, you need to setup the basics for the scene, such as lighting and ground plane where the virtual dominos will be placed. Usually, you also need to setup a camera for the scene, but since you will setup the marker tracking system later on, the camera will be created and added to the scene automatically.

NOTE: Before going through this part, we strongly recommend that you go through Tutorial 1 and Tutorial 5 that comes with the Goblin XNA distribution.

Add Lighting Sources

We will add two directional lights to the scene, each coming from different directions. We also set the ambient light color to a dark gray color so that none of the objects in the scene will appear totally black.

C#

// Create a directional light source
LightSource lightSource = new LightSource();
lightSource.Direction = new Vector3(-1, -1, -1);
lightSource.Diffuse = Color.White.ToVector4();
lightSource.Specular = new Vector4(0.6f, 0.6f, 0.6f, 1);

LightSource lightSource2 = new LightSource();
lightSource2.Direction = new Vector3(1, -1, -1);
lightSource2.Diffuse = Color.White.ToVector4();
lightSource2.Specular = new Vector4(0.6f, 0.6f, 0.6f, 1);

// Create a light node to hold the light source
LightNode lightNode = new LightNode();
lightNode.AmbientLightColor = new Vector4(0.3f, 0.3f, 0.3f, 1);
lightNode.LightSources.Add(lightSource);
lightNode.LightSources.Add(lightSource2);

// Add this light node to the root node
scene.RootNode.AddChild(lightNode);

Create a Ground Plane

Since the game will have physics simulation, we will need a ground plane where the dominos will be placed. We will create a GeometryNode to hold the cubic geometry that represents a flat ground plane, and assign physics properties to this ground. Since we want to show the game board rather than this virtual ground plane, we set IsOcculder property to true. If this property is set to true, this geometry won't be rendered in the frame buffer; rather, it will be used for z-buffer testing. This means that you won't see this ground plane, but if any virtual objects go behind this ground plane, those virtual objects won't be rendered. Since this ground plane won't be visible, we won't assign any materials to it. Finally, we add this ground to the marker node that tracks the game board.

C#

GeometryNode groundNode = new GeometryNode("Ground");
groundNode.Model = new Box(129.5f, 99, 0.2f);

groundNode.Physics.Collidable = true;
groundNode.Physics.Shape = ShapeType.Box;
groundNode.Physics.MaterialName = "Ground";
groundNode.Physics.Pickable = true;
groundNode.AddToPhysicsEngine = true;

groundNode.IsOccluder = true;

groundNode.Model.ReceiveShadows = true;

markerNode.AddChild(groundNode);

Step 2. Setup Capture Device and Marker Tracker

We need to have a video capture device (e.g., a webcam) to capture the real-world environment, and an optical marker tracking system to track the marker array printed on the game board.

NOTE: Before going through this part, we strongly recommend that you go through Tutorial 8, which comes with the Goblin XNA distribution.

Setup Capture Device

Since we're using a webcam to capture the live video, we'll use the DirectShowCapture class. We then initialize this capture device with deviceID = 0, 30Hz framerate, and 640-by-480 resolution, and attach it to our scene. Since we want to show the live video stream in the background, we set ShowCameraImage to true.

If your computer has only one webcam attached (including the embedded cameras on laptop computers), then using deviceID = 0 will work fine. However, if you have more than one webcam, then you need to change this ID to indicate which webcam you want to use.

C#

DirectShowCapture captureDevice = new DirectShowCapture();
captureDevice.InitVideoCapture(0, FrameRate._30Hz, 
    Resolution._640x480, ImageFormat.R8G8B8_24, false);

scene.AddVideoCaptureDevice(captureDevice);

scene.ShowCameraImage = true;

Setup Optical Marker Tracking System

We will use the ALVAR optical marker tracking library from VTT (http://virtual.vtt.fi/virtual/proj2/multimedia/alvar.html). We initialize the tracker with the size of the image we'll be passing for the tracker to analyze, the camera calibration file generated by SampleCamCalib project that comes with ALVAR (you should perform calibration for each of your webcams in order to achieve optimal tracking performance), and the pixel size of the square marker (9.0 means that if you overlay a cube with size of 9x9x9 pixels, then the cube will overlay exactly on top of the marker square). Finally, we assign this tracking system to our scene.

C#

ALVARMarkerTracker tracker = new ALVARMarkerTracker();
tracker.MaxMarkerError = 0.02f;
tracker.InitTracker(captureDevice.Width, captureDevice.Height, "calib.xml", 9.0);
scene.MarkerTracker = tracker;

Create a MarkerNode To Track Gameboard

Since we want to overlay the dominos on top of a game board with marker arrays printed on it, we need to create a MarkerNode that holds the information of the marker configuration we want to track. Then, any geometry attached to this node will be overlaid on top the game board. We create a MarkerNode by denoting which marker tracker will be used to track this marker, the marker configuration file, and a list of marker IDs in this marker configuration. Finally, we add this node to our scene's root node.

C#

// Create a marker node to track the ground plane
int[] ids = new int[54];
for (int i = 0; i < ids.Length; i++)
    ids[i] = i;

markerNode = new MarkerNode(
scene.MarkerTracker, "ARDominoALVAR.txt", ids);

scene.RootNode.AddChild(markerNode);

Step 3. Create 3D Dominos and Balls

Now, we are ready to add 3D objects on top of the game board.

Create 3D Dominos

We'll add textured mapped cubic geometries that represent dominos, and lay them out in a circle. We also need to assign appropriate physics properties to each domino so that they will behave correctly in the physics simulation. Finally, we add all dominos to the marker node that tracks the game board.

C#

dominoModel = new DominoBox(
    new Vector3(dominoSize.X, dominoSize.Z, dominoSize.Y),
    new Vector2(0.663f, 0.707f));

dominoModel.CastShadows = true;
dominoModel.ReceiveShadows = true;

float radius = 18;
for (int x = 0; x < 360; x += 30)
{
    GeometryNode dominoNode = 
        new GeometryNode("Domino " + dominos.Count);
    dominoNode.Model = dominoModel;

    dominoNode.Physics.Mass = 20;
    dominoNode.Physics.Shape = ShapeType.Box;
    dominoNode.Physics.MaterialName = "Domino";
    dominoNode.Physics.Pickable = true;
    dominoNode.AddToPhysicsEngine = true;

    Material dominoMaterial = new Material();
    dominoMaterial.Diffuse = Color.White.ToVector4();
    dominoMaterial.Specular = Color.White.ToVector4();
    dominoMaterial.SpecularPower = 10;
    dominoMaterial.Texture = cguiLogoTexture;

    dominoNode.Material = dominoMaterial;

    TransformNode dominoTransNode = new TransformNode();
    dominoTransNode.Translation = new Vector3(
        (float)(radius * Math.Cos(MathHelper.ToRadians(x))),
        radius * (float)(Math.Sin(MathHelper.ToRadians(x))), 
        dominoSize.Y / 2);
    dominoTransNode.Rotation = 
        Quaternion.CreateFromAxisAngle(Vector3.UnitZ, 
            MathHelper.ToRadians(x + 90));

    markerNode.AddChild(dominoTransNode);

    dominoTransNode.AddChild(dominoNode);

    dominos.Add(dominoNode);
}

Create 3D Balls

We will create a list of 3D balls and add them to the scene even before the player shoots the ball in order to have best performance. Since we don't want to show these balls yet, we'll add them outside of the view frustum. When the player shoots a ball, we'll simply pick a ball from this list and give it a new position and velocity. Since the code is quite similar to creating 3D dominos, we will omit the codes here (see CreateBalls() function from line 1459)

Step 4. Setup Physics Simulation and Sound Effect

It is very easy to setup the physics simulation using Goblin XNA as well as use a sound effect when two objects collide into each other. The physics properties set to the GeometryNode in the previous steps are used by the physics engine. There are many properties associated with GeometryNode.Physics, but there are only few properties that you really have to define to make the simulation work.

C#

dominoNode.Physics.Mass = 20;
dominoNode.Physics.Shape = ShapeType.Box;
dominoNode.Physics.MaterialName = "Domino";
dominoNode.Physics.Pickable = true;
dominoNode.AddToPhysicsEngine = true;

Taking the domino geometry as an example, we need to define its mass, the shape primitive used in the physics simulation, the material name is necessary only if we're going to modify its physics material later on, which is the case for this game), and whether this geometry can be picked (we want to pick and manipulate the dominos in certain game modes, so we set this to true). Actually, we also need to set Collidable and Interactable properties to true for the geometry to collide and have physical response when other geometry hits this geometry. However, the game mode starts with ‘Add' mode, which will be explained later, and we don't want the geometry to collide and react on the collision when we move the selected domino around on the game board using mouse. So we leave them as false when we create them, and then set those two properties to true once the game starts.

NOTE: Before going through this part, we strongly recommend that you go through Tutorial 5 and Tutorial 9, which come with the Goblin XNA distribution.

Setup Physics Simulation

Since Goblin XNA supports Newton dynamics library, we will use NewtonPhysics implementation for our physics simulation. The default simulation space defined by Newton library is 200x200x200 centered at the origin, and if any objects go outside of this bound, those objects won't be simulated for optimization purposes. Since our scene is larger than this default space size, we'll set it to 500x500x500 centered at the center. The default gravity is 9.8, but this is quite slow for a realistic simulation, so we'll increase the gravity to 30. We will also increase the maximum simulation sub-steps to 5 so that even if the frame rate gets below 60 FPS, the physics engine will simulate as many as 5 sub-steps to simulate at least 60 FPS.

C#

scene.PhysicsEngine = new NewtonPhysics();
// Make the physics simulation space larger 
// to 500x500x500 centered at the origin
((NewtonPhysics)scene.PhysicsEngine).WorldSize =
    new BoundingBox(Vector3.One * -250, Vector3.One * 250);
// Increase the gravity
scene.PhysicsEngine.Gravity = 30.0f;

((NewtonPhysics)scene.PhysicsEngine).MaxSimulationSubSteps = 5;

Setup Physics Material and Sound Effect

We will setup some material properties for each physics entity that define how each entity will behave when they collide. Each entity is defined by a material name (for example, “Domino” for domino geometries), and in the example code below, this material definition is applied between “Ground” and “Ball” entities. For detailed explanation of each material properties (e.g., elasticity, friction, etc), please see the user manual that comes with the Goblin XNA distribution. When we set the material properties between two physics entities, we can also define a callback function, which will be invoked when the two entities collide. For our game, we want to add a sound effect to indicate the collision. For better performance, we limit the number of concurrent sound effects, and we only play a sound effect if the contact speed between the two entities is greater than certain speed (e.g., in the code example below, it will only play sound effect if the contact speed is above 3 pixels/sec). We also change the volume of the sound effect based on its contact speed.

C#

NewtonMaterial physMat2 = new NewtonMaterial();
// Gound to ball material interaction
physMat2.MaterialName1 = "Ground";
physMat2.MaterialName2 = "Ball";
physMat2.Elasticity = 0.7f;
physMat2.ContactProcessCallback = delegate(Vector3 contactPosition, 
    Vector3 contactNormal, float contactSpeed, 
    float colObj1ContactTangentSpeed, float colObj2ContactTangentSpeed,
    Vector3 colObj1ContactTangentDirection, 
    Vector3 colObj2ContactTangentDirection)
{
    if (contactSpeed > 3f)
    {
        if (soundsPlaying.Count >= SOUND_LIMIT)
            return;

        try
        {
            Sound.SetVolume("Default", contactSpeed * volume);
        }
        catch (Exception exp) { }
        try
        {
            Sound.Play("rubber_ball_01");
            soundsPlaying.Add(DateTime.Now.TimeOfDay.TotalMilliseconds);
        }
        catch (Exception exp) { }
    }

};
((NewtonPhysics)scene.PhysicsEngine).AddPhysicsMaterial(physMat2);

The above code only defines the material properties between “Ground” and “Ball” entities. In the actual game code, however, it defines the material properties for each pair of existing physics entities.

Step 5. Add User Interaction System

Now, we have all of the components necessary to start playing the AR domino knock-down game except for a way of interacting with the game for the player. Ideally, we want to play the game using a touch-sensitive mobile device with an embedded camera, such as an iPhone or an Ultra-Mobile-PC. Due to platform restriction and the high cost of those devices, we will use a webcam with mouse and keyboard based interaction for this demo. But first, we will add several callback functions for mouse and keyboard events.

C#

MouseInput.Instance.MousePressEvent += 
    new HandleMousePress(MousePressHandler);
MouseInput.Instance.MouseDragEvent += new HandleMouseDrag(MouseDragHandler);
MouseInput.Instance.MouseReleaseEvent += 
    new HandleMouseRelease(MouseReleaseHandler);
MouseInput.Instance.MouseMoveEvent += new HandleMouseMove(MouseMoveHandler);

KeyboardInput.Instance.KeyPressEvent += new HandleKeyPress(KeyPressHandler);
KeyboardInput.Instance.KeyReleaseEvent += 
    new HandleKeyRelease(KeyReleaseHandler); 

For this game, we have implemented three different modes, “Add”, “Edit”, and “Play”.

Add Mode

In “Add” mode, we want the players to be able to add dominoes anywhere on the game board by clicking the mouse and dragging the newly added domino around to finalize its position. We also want the players to be able to rotate the added domino along the normal vector of the game board by pressing “Left” and “Right” arrow keys.

NOTE: The following codes are in the MousePressHandler function.

C#

Vector3 nearSource = 
    new Vector3(mouseLocation.X, mouseLocation.Y, 0);
Vector3 farSource = 
    new Vector3(mouseLocation.X, mouseLocation.Y, 1);

Matrix viewMatrix = 
    markerNode.WorldTransformation * State.ViewMatrix;

Vector3 nearPoint = 
    graphics.GraphicsDevice.Viewport.Unproject(
        nearSource, State.ProjectionMatrix, 
        viewMatrix, Matrix.Identity);
Vector3 farPoint = 
    graphics.GraphicsDevice.Viewport.Unproject(farSource, 
        State.ProjectionMatrix, viewMatrix, Matrix.Identity);
// Cast a ray to the scene to pick a point on the ground where 
// the domino will be added
List<PickedObject> pickedObjects = 
    ((NewtonPhysics)scene.PhysicsEngine).PickRayCast(nearPoint, farPoint);

for (int i = 0; i < pickedObjects.Count; i++)
{
    if (((GeometryNode)pickedObjects[i].PickedPhysicsObject.Container).
        Name.Equals("Ground"))
    {
        Vector3 intersectPoint = nearPoint * (1 – 
            pickedObjects[i].IntersectParam) +
            pickedObjects[i].IntersectParam * farPoint;

        dominoTransNode.Translation = intersectPoint + Vector3.UnitZ * 
            dominoSize.Y / 2;
        break;
    }
}

Before we add a domino, we need to figure out the 3D point on the gameboard that corresponds to the 2D mouse click point on the screen. To do this, we need to create a 3D ray originating from the 3D point on the near clipping plane to the 3D point on the far clipping plane, both of which correspond to the clicked 2D point on the screen. XNA's Viewport class supports the Unproject function that performs this 2D to 3D mapping, which requires the 3D plane to map to (in this case, it's the nearSource and farSource), the projection matrix, the view matrix, and the world matrix. The tricky part here is the concept of the computation of the view matrix. In Goblin XNA, to make things simple, we decided not to have the marker transformation affect the physics simulation by default, so even if the marker transformation changes, nothing changes in the physics simulation world. However, if we want to figure out the actual view matrix in the physics simulation world, we need to multiply the marker transformation with the current view matrix, which is the State.ViewMatrix.

Once we get the 3D ray, we simply pass it to Newton library to perform the picking and then it will return a list of intersecting objects. We then check to see if there is an intersecting geometry named “Ground,” which is the gameboard. If the game board is intersected, then we calculate the 3D intersection point on the game board and put the newly added domino in that intersection point. In the MouseDragHandler function, you will see similar areas of code that let the player drag the newly added domino around on the game board.

There are also codes that let the player rotate the new domino along the normal vector of the game board in KeyPressHandler and Update functions, but we will omit the explanation here since the codes are self-explanatory.

Edit Mode

In “Edit” mode, we want the players to be able to pick an existing domino on the gameboard and drag it around. We also want the players to be able to rotate the selected domino along the normal vector of the game board by pressing “Left” or “Right” arrow keys.

NOTE: The following codes are in the MousePressHandler function.

C#

// Cast a ray to the scene to select the closest 
// object (to the eye location) that hits the ray
List<PickedObject> pickedObjects =   
    ((NewtonPhysics)scene.PhysicsEngine).PickRayCast(nearPoint, farPoint);

// If one or more objects intersect with our ray vector
if (pickedObjects.Count > 0)
{
    // Sort the picked objects in ascending order of intersection parameter
    // (the closest to the eye is the first one in the sorted list)
    pickedObjects.Sort();

    // If the closest object is the ground, then don't do anything
    if (((GeometryNode)pickedObjects[0].PickedPhysicsObject.Container).
        Name.Equals("Ground"))
        return;

    selectedDominos.Add(
        (GeometryNode)pickedObjects[0].PickedPhysicsObject.Container);
    
    GeometryNode domino = selectedDominos[0];
    Vector4 mod = domino.Material.Diffuse;
    domino.Material.Diffuse = 
        new Vector4(mod.X, mod.Y, mod.Z, 0.5f);
}

The picking codes are the same as before, but now we want to pick an existing domino instead of the game board. Since there are multiple dominoes on the game board, it is possible that the picking ray intersects with more than one domino (and most likely the game board as well). Thus, we need to sort the intersected geometries based on their distances from the camera, and then we want to select the one closest to the camera. If the closest geometry is not the game board, then we add this domino to the list of selected dominoes and set its transparency to half-transparent to indicate the selection. Then, in the MouseDragHandler function, we will see similar codes with “Add” mode that let the player drag the selected domino around.

Play Mode

In “Play” mode, we want the players to be able to shoot balls from the clicked location. We allow the player to shoot either heavy or normal balls depending on whether the click was on the right or left mouse button.

NOTE: The following code snippets are in the MouseReleaseHandler function since we allow the player to inject additional speed when pushing the camera forward while press-and-holding the mouse button, and then releasing the button (this interaction makes more sense when the game is played on an UMPC).

C#

bool heavy = (button == MouseInput.RightButton);

float additionalSpeed = 0;

// Add extra speed to the ball being shot if the camera point is moved from 
// far point to closer point (to the ground)
float ballReleaseDistance = 
    markerNode.WorldTransformation.Translation.Length();

if (ballReleaseDistance < ballPressDistance)
    additionalSpeed = ballPressDistance - ballReleaseDistance;

Vector3 nearSource = 
    new Vector3(uiManager.CrossHairPoint.X, 
        uiManager.CrossHairPoint.Y, 0);
Vector3 farSource = 
    new Vector3(uiManager.CrossHairPoint.X, 
        uiManager.CrossHairPoint.Y, 1);

Matrix viewMatrix = markerNode.WorldTransformation * State.ViewMatrix;

Vector3 nearPoint = 
    graphics.GraphicsDevice.Viewport.Unproject(nearSource,
        State.ProjectionMatrix, viewMatrix, Matrix.Identity);
Vector3 farPoint = 
    graphics.GraphicsDevice.Viewport.Unproject(farSource,
        State.ProjectionMatrix, viewMatrix, Matrix.Identity);

ballCount++;
uiManager.BallCount++;
// Create a ball and shoot it
CreateBallCharactor(nearPoint, farPoint, additionalSpeed, heavy);

C#

Vector3 linVel = far - near;
linVel.Normalize();

GeometryNode ballNode = (heavy) ? heavyBalls[curHeavyBall] : balls[curBall];
ballNode.Model.CastShadows = true;
ballNode.Model.ReceiveShadows = true;
Vector4 orig = ballNode.Material.Diffuse;
ballNode.Material.Diffuse = 
    new Vector4(orig.X, orig.Y, orig.Z, 1);

Vector3 v = near + linVel * 10;

// Forces the physics engine to 'transport' 
// this ball to the location 'v' in the simulation world
((NewtonPhysics)scene.PhysicsEngine).SetTransform(
    ballNode.Physics, Matrix.CreateTranslation(v));

// Apply a linear velocity to this ball
((NewtonPhysics)scene.PhysicsEngine).ApplyLinearVelocity(
    ballNode.Physics, linVel * (50f + additionalSpeed));

if (heavy)
{
    curHeavyBall++;
    // Circles through if we run out of heavy balls
    if (curHeavyBall >= BALL_NUM / 2)
        curHeavyBall = 0;
}
else
{
    curBall++;
    // Circles through if we run out of normal balls
    if (curBall >= BALL_NUM / 2)
        curBall = 0;
}

Step 6. Add Heads-Up-Display (HUD)

Finally, we add a HUD for displaying information such as elapsed game time, number of balls used, and a victory message when the game is over. We also add a cross hair to indicate the position of the mouse on the screen.

NOTE: The above codes are in the Draw function of UIManager class.

C#

// Count the time elapsed in the play mode if game is not over yet
if (gameState.CurrentGameMode == GameState.GameMode.Play &&  
    !gameState.GameOver)
{
    gameState.ElapsedSecond += gameTime.ElapsedRealTime.TotalSeconds;
    if (gameState.ElapsedSecond >= 60)
    {
        gameState.ElapsedMinute++;
        gameState.ElapsedSecond = 0;
    }
    // Update the time elapsed label with the new elapsed time
    modeStatusLabel = "Play: " + (int)gameState.ElapsedMinute + ":" +  
        String.Format("{0:D2}", (int)gameState.ElapsedSecond) + "  ";
}

// Draws the cross hair mark to indicate the mouse position which is used for
// ball shooting, domino addition, domino edition, and GUI interaction (if 
// GUI is shown)
UI2DRenderer.FillRectangle(new Rectangle(crossHairPoint.X - 10, 
    crossHairPoint.Y, 23, 3), null, new Color(50, 205, 50, 150));
UI2DRenderer.FillRectangle(new Rectangle(crossHairPoint.X, crossHairPoint.Y -  
    10, 3, 23), null, new Color(50, 205, 50, 150));

UI2DRenderer.WriteText(Vector2.Zero, modeStatusLabel, 
    Color.White, labelFont,
    GoblinEnums.HorizontalAlignment.Right, 
    GoblinEnums.VerticalAlignment.Top);

// Displays how many balls are used so far
if (gameState.CurrentGameMode == GameState.GameMode.Play)
{
    UI2DRenderer.WriteText(Vector2.Zero, 
        "Balls Used: " + ballCount, Color.Red, 
        labelFont, GoblinEnums.HorizontalAlignment.Right,       
        GoblinEnums.VerticalAlignment.Bottom);
}

// Shows the victory texts as well as play the victory sound if won
if (gameState.GameOver)
{
    UI2DRenderer.WriteText(new Vector2(0, 130), "Victory!! You Won!!", 
        Color.Red, victoryFont, GoblinEnums.HorizontalAlignment.Center, 
        GoblinEnums.VerticalAlignment.None);

    UI2DRenderer.WriteText(new Vector2(0, 180), 
        "Time: " + (int)gameState.ElapsedMinute + ":" + 
        String.Format("{0:D2}", (int)gameState.ElapsedSecond), 
        Color.Red,victoryFont, 
        GoblinEnums.HorizontalAlignment.Center, 
        GoblinEnums.VerticalAlignment.None);

    UI2DRenderer.WriteText(new Vector2(0, 220), 
        "Balls Used: " + ballCount, Color.Red, 
        victoryFont, GoblinEnums.HorizontalAlignment.Center, 
        GoblinEnums.VerticalAlignment.None);

    Texture2D trophy = null;
    if (gameState.ElapsedMinute < 1 && 
        gameState.ElapsedSecond <= 14)
        trophy = trophyGold;
    else if (gameState.ElapsedMinute < 1 && 
        gameState.ElapsedSecond <= 25)
        trophy = trophySilver;
    else
        trophy = trophyBronze;

    UI2DRenderer.FillRectangle(new Rectangle(
        (State.Width - trophy.Width) / 2, 280, 
        trophy.Width, trophy.Height), 
        trophy, Color.White);
}

Conclusion

This article demonstrated how to implement a simple yet interesting interactive AR game. Since this article is rather advanced, and most of the codes are specific to Goblin XNA framework, you will have a better understanding of the source codes if you go through some of the suggested tutorials first. If you have an UMPC (Ultra-Mobile-PC) with you, then you can also run the same code, and you can shoot the balls by tapping on the screen rather than using a mouse.

If you want to try this out, download the source code via the link at the top of the article!

About The Author

Ohan Oda is a Ph.D student at Columbia University studying Computer Science. He specializes in designing systems and user interaction techniques for entertainment in Augmented Reality. He is the lead developer of Goblin XNA framework, and has worked on several Augmented Reality projects using this framework. Please contact ohan@cs.columbia.edu for any questions.

Introduction

Back in November of 2009, I was met with an interesting challenge. Those of you who frequent the XNA Community Forums have probably heard of Nick Gravelyn. He was an XNA MVP, and now he works for the XNA team. He started a little contest called xna7day, which was designed to challenge developers to make a game in 7 days using a pre-defined theme.

The theme in November was “Text Based” and its rule was that you were only allowed to use text in the visuals of your game. I saw some of the cool stuff other developers were working on, including one that looked like it had text characters that could walk and talk, like people. But my mind immediately went to a different solution: I wanted to make a game that was fully 3D (effectively breaking the rule of rendering text), but then post-process the render into ASCII art, so it would look like it was text.

In this article, I'm going to show you how to do just that:

Setup

Alright, let's dive right in. Open up the base project. It should compile and run as is. It's a simple, if ugly, first-person shooter. Play through the first level. Go ahead, I'll wait. Bonus points to anyone who recognizes the level design!

Okay, got a playthrough? Let's start! I separated this into three projects so the ASCII stuff can be extracted into other games. We'll be doing most of our work in the ASCII_Renderer project, so let's open that up and add a new class to the Source\ascii folder called “Renderer.” I use the namespace “ASCII3D” for all files in this project, so get rid of the extra folder sub-namespaces. Add these using statements, too:

C#

using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.Graphics; 

This is going to be a post-processing effect, so we'll try to make it portable. Add the following empty function declarations:

C#

public Renderer(ContentManager content, int xRes, int yRes)
{ }

public void StartScene()
{ }

public void EndScene()
{ }

The constructor takes three arguments: The ContentManager, which we will use to load in our font and post-processing shader, and the resolution variables, which will basically tell us how many lines of text to draw and how many characters in each line. Since we have these prototyped, we'll make our only edits to the ASCIIFPS project in the Source\Game1.cs file. But first, we need to define and initialize a new Renderer object. Add a new class variable:

C#

/// <summary>
/// This is what handles the ASCII conversion
/// </summary>
private Renderer renderer; 

And at the end of the LoadContent function:

C#

// and initialize our ASCII renderer
// the variables we pass make each character
// appx 10x14 which seems to be the smallest
// readable size
renderer = new Renderer(Content, Global.ScreenWidth/10,Global.ScreenHeight/14);

Finally, call the BeginScene and EndScene functions. Modify Game1's Draw function to look like this:

C#

protected override void Draw(GameTime gameTime)
{
    // notice we only wrap StartScene and
    // EndScene if we are UsingASCII
    if (UsingASCII)
        renderer.StartScene();

    currentState.Draw(gameTime);

    if (UsingASCII)
        renderer.EndScene();

    base.Draw(gameTime);
}

The Boolean variable UsingASCII is defined for us. It defaults to false and can be toggled any time in-game by pressing the [M] key. If you want to default it to true for testing purposes, just change its value in Game1's Initialize function. At this point, the Renderer is all hooked up, so changes will go right in.

I skimmed over the other classes earlier in the article, but I need to touch on a few of them. Almost everything in GlobalModules is fairly standard utility stuff. We'll be using it, so you might want to glance at the Global class. The ASCIIFPS has the FPS engine in it and is almost entirely irrelevant to this article, except that it offers a fun test-bed for our effect. Feel free to go through the code, but I'll warn you: I wrote it for a 7-day contest, so some of it looks like gibberish and is only incidentally functional. Finally, we have the ASCII_Renderer project. This already has two files in it: BitmapFontGenerator and Letter. The latter are for generating a bitmap font map that we'll use in a bit.

The functionality isn't specific to this article, since any artist could easily make this. But I'm no artist, which is why I had the computer do it for me. Basically, BitmapFontGenerator has a static function that will take a texture of letters (see the Content project in ASCII_Renderer), trim them, sort them by how much they fill their space, and print them into a new Texture for our functions. Feel free to read through the code for this process; I commented it pretty well.

Post-Processing Basics

We need to cover the basics of post-processing before we start coding the next part. When post-processing a scene, you first render the entire scene off-screen on a RenderTarget. Then, you can apply any effects you want to the resolved texture before putting it on the screen. That's it. Here's how in XNA:

C#

public class Renderer
{
    private RenderTarget2D SrcImage;

    public Renderer(ContentManager content, int xRes, int yRes)
    {
        SrcImage = new RenderTarget2D(Global.Graphics, 
            xRes, yRes, 1, SurfaceFormat.Color);
    }

    public void StartScene()
    {
        Global.Graphics.SetRenderTarget(0, SrcImage);
        Global.Graphics.Clear(Color.Black);
    }

    public void EndScene()
    {
        Global.Graphics.SetRenderTarget(0, null);
        Global.Graphics.Clear(Color.Black);

        // TODO: draw the texture from the RenderTarget
    }
}

We create the RenderTarget2D we need in the constructor. Notice that the size of the RenderTarget is the same as the number of ASCII characters we plan to draw on screen. I'll explain that in the next section. In the StartScene function, we set the RenderTarget on the device so that all draw calls will actually draw to it. Then we clear the buffer, because RenderTargets' content from frame to frame is undefined. In EndScene, the RenderTarget is set to null, which is DirectX's way of saying “use the screen's framebuffer.” Now we need to actually draw the rendered image. XNA requires shaders for all draw calls, even if they're veiled by BasicEffect and SpriteBatch. We need to create a very simple shader to draw the texture to screen. Create a new Effect file in ASCII_Renderer's Content project and name it TextEffect. We'll start it out with this:

HLSL

texture2D sourceTex;
sampler2D SourceTextureSampler = sampler_state
{
    Texture = <sourceTex>;
    MinFilter = point;
    MagFilter = point;
    MipFilter = point;
    AddressU = wrap;
    AddressV = wrap;
};

struct VertexShaderInput
{
    float4 Position : POSITION0;
    float2 TexCoords : TEXCOORD0;
};

struct VertexShaderOutput
{
    float4 Position : POSITION0;
    float2 TexCoords : TEXCOORD0;
};

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
{
    VertexShaderOutput output;
    
    output.Position = input.Position;
    output.TexCoords = input.TexCoords;

    return output;
}

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
    float4 sourceColor = tex2D(SourceTextureSampler,input.TexCoords);
    
    return sourceColor;
}

technique Technique1
{
    pass Pass1
    {
        VertexShader = compile vs_2_0 VertexShaderFunction();
        PixelShader = compile ps_2_0 PixelShaderFunction();
    }
}

There are a couple things I want to note about the base shader. First, notice that we use point filtering. I'll explain why in the next section, but it's important to see where we did it. Next, notice that we took out all the default transformation matrices. We're going to be drawing a single full screen quad, so we can just pass in the coordinates in screen-space—no processing required. Now we need to make a few changes to the Renderer class to use this Effect and draw the rendered game. First, we need to add some new class variables:

C#

private Effect effect;

private VertexPositionTexture[] verts;
private VertexDeclaration vd;

The vertices are going to be for a simple quad that takes up the whole screen. They should be initialized in the LoadContent function like this:

C#

effect = content.Load<Effect>("TextEffect");

verts = new VertexPositionTexture[6];
verts[0] = new VertexPositionTexture(
    new Vector3(-1, -1, 0), new Vector2(0, 1));
verts[1] = new VertexPositionTexture(
    new Vector3(1, -1, 0), new Vector2(1, 1));
verts[2] = new VertexPositionTexture(
    new Vector3(-1, 1, 0), new Vector2(0, 0));
verts[3] = verts[1];
verts[4] = verts[2];
verts[5] = new VertexPositionTexture(
    new Vector3(1, 1, 0), new Vector2(1, 0));

vd = new VertexDeclaration(
    Global.Graphics, VertexPositionTexture.VertexElements);

And finally, we add our draw code to the Draw function:

C#

effect.Begin();
effect.CurrentTechnique.Passes[0].Begin();

effect.Parameters["sourceTex"].SetValue(SrcImage.GetTexture());
effect.CommitChanges();
Global.Graphics.RenderState.CullMode = CullMode.None;

Global.Graphics.VertexDeclaration = vd;
Global.Graphics.DrawUserPrimitives<VertexPositionTexture>
     (PrimitiveType.TriangleList, verts, 0, 2);

effect.CurrentTechnique.Passes[0].End();
effect.End();

We set the texture we got from the RenderTarget, turn off culling (what would be culled in this geometry?), and draw it. Run the project. You should be able to toggle a difference in-game with the [M] key.

Drawing Some Text

“Okay,” you say, “it runs at a lower resolution with the [M] key. So what?” Remember what we set the resolution to in the RenderTarget? Yes, every pixel on this low-resolution version will be a character in the final image. How do we turn these big pixels into characters? First we need to define a texture bitmap font. Let's add a Texture2D class variable to Renderer and a const int for the BitmapFontGenerator:

C#

public Texture2D text; 
private const int NUM_SLOTS = 256;

Then we'll use the BitmapFontGenerator to construct this texture:

C#

text = BitmapFontGenerator.GenerateTextTexture(content, NUM_SLOTS);

Okay, that's all for now. Let's go back to the TextEffect.fx. We need to add a few variables:

HLSL

float2 dstLetterSize;

int numLetters;

texture2D textTex;
sampler2D TextTextureSampler = sampler_state
{
    Texture = <textTex>;
    MinFilter = point;
    MagFilter = point;
    MipFilter = point;
    AddressU = wrap;
    AddressV = wrap;
};

The variable “dstLetterSize” will tell us how big the letters on the screen are. The “numLetters” variable will tell us how many letters are in the font texture. And then there's the font texture itself. Here's how we draw actual characters in the Pixel Shader:

HLSL

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
     float4 sourceColor = tex2D(SourceTextureSampler,input.TexCoords);
    
     float2 texCoords = float2(
          input.TexCoords.x-((int)
               (input.TexCoords.x/dstLetterSize.x)*dstLetterSize.x),
          input.TexCoords.y-((int)
               (input.TexCoords.y/dstLetterSize.y)*dstLetterSize.y));
     texCoords /= dstLetterSize;
     texCoords.x /= numLetters;
    
     float4 letterColor = tex2D(TextTextureSampler,texCoords);
    
     return letterColor;
}

We've already seen the source color line. The next line will get the offset into the letter in letter size by subtracting the rounded-down (or integer-casted) texture coordinates scaled down. The line after that scales it back up to a 0-1 range. So now we have the new variable texCoords that is filled with the texture coordinates of the letter in 0-1 terms. We divide the x by numLetters because our texture is one long horizontal line of letters. Finally we get the letter color with these new texture coordinates. This letter color is grayscale, and when multiplied by the source color, we've assured that each letter is rendered in its own slot with the color from the original 3D render. The last thing we have to do is set those effect variables we defined from Renderer's EndScene function. Where we once just set the source texture, we now set these:

C#

effect.Parameters["dstLetterSize"].SetValue(
     new Vector2(1.0f/(float)SrcImage.Width, 1.0f/(float)SrcImage.Height));
effect.Parameters["numLetters"].SetValue(NUM_SLOTS);
effect.Parameters["sourceTex"].SetValue(SrcImage.GetTexture());
effect.Parameters["textTex"].SetValue(text);
effect.CommitChanges();

Most of these are self explanatory, but the dstLetterSize is a bit more complicated. To get the size of the letter in the 0-1 texture space, you take the total size (0 to 1) divided by the number of letters in that axis which, in our case, is the RenderTarget size.

Wheel of Fortune

‘M's aren't very interesting. In traditional ASCII art, the whole point is that you use characters that take more space or less space to handle shading. Let's add this code after the multiplication of the texCoords.x:

HLSL

float lum = (sourceColor.r+sourceColor.g+sourceColor.b)/3.0; 
float val = max(fullValueColor.r,
     max(fullValueColor.g,fullValueColor.b));
    
int ind = ((numLetters-1)-
     max(0,min((numLetters-1),(int)(lum*numLetters))));

texCoords.x += ind*(1.0/numLetters);

The first line gets the luminosity of the source color. The next gets the value. The next line gets the letter index from the luminosity. Subtracting it from (numLetters-1) reverses the index, because we want luminosity=1 to equate to index=0, because that has the letter that takes up the most space. Run the game. You'll see it now shows value, but it's still pretty hard to tell what's on the screen:

The last thing we can do is colorize it. You could just multiply it by the source color and get something like this…

…but everything looks too dark because the source color still has its value. Plus we're already practically multiplying it by choosing the letter. So we need to get the full value color first:

HLSL

float4 fullValueColor = sourceColor * (1.0/val);

We just multiply the original color by the inverse of the value. And then multiply that fullValueColor by the letter color in the return:

HLSL

return fullValueColor * letterColor;

That should produce a nice result:

Conclusion

And there it is! We have a real-time 3D render transformed into ASCII art. It's difficult to see and it's not very useful, but it's sure neat, eh? Why not add it as an unlockable easter egg in your next XBL Indie Game? I'm looking forward to seeing how people use this effect! If you use it, please send me an email with a screenshot!

About The Author

Louis Ingenthron is a Game Developer in Orlando, FL. He works on commercial Console and PC titles, but runs his own Indie Games company, FV Productions, on the side. He is best known for his open source XNA rhythm game Unsigned and specializes in real-time Graphics programming. He has been working with XNA since the 2.0 Beta and with .NET C# for just as long. He is also familiar with several other development languages, such as C, C++, and Java. Occasionally, he can even be found doing web development and Flash. Louis also writes for MSDN's Coding4Fun website, contributing articles on a monthly basis.