• [0:42] March CTP (“CTP4”) of Visual Studio Update 2 now available (Charles Sterling)
• [2:07] Now Available: Office Developer Tools for Visual Studio 2012 (S. Somasegar)
• [3:33] Visual Studio Tools for Git 0.8.0.0 (Andy Lewis)
• [4:49] Announcing a New Preview of the LightSwitch HTML Client
• [6:07] Getting Started with LightSwitch in Visual Studio 2012 Update 2 CTP4 (Beth Massi)
• [6:40] MetroAppSite: Free, Open Source Metro-Style Website Templates for Your Windows Store Apps (Aaron Stannard)
• [7:35] Recording of Webinar 'Applied MVVM in Windows 8 apps' by Gill Cleeren (Gill Cleeren)
• [9:01] Advanced MVVM concepts using Contoso Cookbook: An analysis of a complete MVVM application: Part 1 (Gill Cleeren)
• [9:44] Easy Access to Kinect for Windows Sample Code (Ben Lower)
• [10:34] Interpolation Tricks

Picks of the Week!

• Dan's Pick of the Week: [12:06] Unity Windows Store apps open beta, and Channel 9 Live event!
• Brian's Pick of the Week: [12:52] Panoramic Camera Head

Developing FallFury was a fun and educational activity. Besides the fact that I got to work with an amazing team of people who were willing to assist me with the testing and development, I learned some important points about the development process as a whole and have outlined them in this article.  Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-12-Project-Conclusion.  For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

Go Outside Your Comfort Zone

This past summer, I decided to tackle a new challenge—game development. Never before had I built a game from the ground up. Moreover, as I could already find my way around C# code, I decided that I wanted to make C++ the language of choice, and so worked with DirectX instead of XNA. Additionally, the project needed to be constructed with Windows 8 as the target platform and I’d have to rely on the WinRT stack.

The concepts were new to me, but I figured that learning about a new platform and a new language would be more interesting than sticking to my comfort zone. Through FallFury I extended my knowledge about pointers, trigonometry, how DirectX handles a lot of the graphics legwork, and how to write HLSL shaders.

It was a challenging but very rewarding experience.

Communicate

The most important part of any project is communication. There are always people who know more than you in one field or another, so don't hesitate to reach out to other developers and ask questions. Your code is not perfect, so talk about best practices used in production and how you can improve on what you already have—communication should not be limited to your team, floor, or even building.

One bonus of my experience at Microsoft was discovering that a lot of developers, when not in a meeting or in the middle of an important assignment, are more than happy to help a fellow employee with a piece of advice or some code review.

Plan the Small Details

Before you even touch the keyboard to write your first line of code, outline the components of your application and have a general idea of how you’ll implement those components as well as how they’ll interact with each other. It is tempting to start coding and then add or modify features, but doing so will create more problems than it will solve.

Build Small Prototypes

FallFury is a complex project composed out of blocks such as:

• XML level/metadata reader and writer
• The Share and Settings charm components
• SpriteBatch sub-system
• Level renderer
• Screen sub-system
• Menu interaction sub-system

Each block is responsible for its own set of tasks and each can be individually tested with either mocked data or a loose connection to the existing codebase. The necessity for small prototypes comes from individual requirements tied to the platform and possible integration scenarios. For example, when I began development, I created a non-hybrid DirectX project, which meant that if I needed to introduce XAML integration, I had to rewrite small parts of my swap chain preparation stack as well as work with a lot of XAML content in the code-behind. The Settings charm integration heavily relied on XAML, and I would have had to write the popup and control-related code in C#.

Obviously, that was not an option, so I created a hybrid project instead and ported most of my code into the new solution. Lesson learned here: prototype—and prototype early—in order to make sure that what you plan to implement will be implemented in an efficient manner.

Test Often, Test with Other People

When working on an application, it is important to understand the needs and expectations of your target audience, and a big part of this is making sure that the application provides a way for the user to easily learn about an action without a helper booklet. The menu system is a good example of this in FallFury. As I was designing the buttons on my screens, I tried to implement the approach used in games such as Dance Central—the user simply slides the button-panel and it takes him to the screen that was linked to that specific button. However, as I was testing the application with the local development crowd, I noticed a pattern—people were attempting to simply tap the button instead of sliding it. To avoid this confusion, I added a pulsating arrow on the right side of the button in order to highlight the fact that the button should be displaced in order to be activated.

The XML level loading system is another example of how secondary testing improved FallFury. Initial element positioning relied on a given level length, as well as on a ration-based placement. For example, if I wanted to put a button in the middle of the screen, I could position it with a value, such as .5. The level loader would then translate this value relative to the screen size. Rick Barraza was creating some test levels when he notified me about a big flaw—as the level extended, some elements were mis-positioned by a large margin compared to the original intended location. This was because when a level length was reset, the ratio was applied on a larger/smaller value, and some elements would therefore overlap. Ultimately, I rewrote the level structure to rely on pixel values, and designed the level-length to calculate automatically based on the included elements and their size.

Test on Different Hardware

FallFury started with portable hardware in mind. Its original design targeted Windows 8 tablet devices, though it was later decided that the project should support desktop machines as well. That automatically added thousands of possible hardware combinations on which the game might potentially run. Here are just some of the test machines we got to use:

• Samsung Series 7 Slate
• ARM Developer Tablet (Microsoft Surface)
• ASUS EP121 Slate

Each of these machines came with its own specific conditions that had to be accommodated. While Samsung Series 7 Slate featured an accelerometer, at the time of development the ASUS EP121 tablet did not have Windows 8 drivers for its sensor. When I sideloaded the game, I noticed that I could no longer control the character the way I wanted to. Also, though FallFury has keyboard controls, I had to develop controls for machines without keyboards. This is why I came up with the concept of touch-based, on-screen controls.

Moving on to the ARM device, first and foremost I noticed that my shader configurations were not compatible with the platform capabilities. All my testing was done on a machine that supported DirectX Feature Level 10, which was not the case for a low-power ARM machine that supports DirectX Feature Level 9.1. I had to work with some MSDN samples to find a way to properly modify the required shaders so they would work on Windows RT. ARM devices are built with battery life in mind. Therefore, their capabilities are reduced compared to those of a full-sized Windows tablet. This had an impact on the general game performance—some parts of the game, such as particle rendering, were working fine on a desktop machine, but were quite laggy on the ARM tablet. Knowing the possible source of the problem, I had to optimize the particle storage and rendering stack to ensure they didn’t affect the user experience on Windows RT hardware.

There Is Never Enough Time - Learn To Cut

When I started planning FallFury, I worked up a huge list of features. As the deadline approached, however, I noticed that I somewhat underestimated the allocated project time—a lot of time was spent on testing and eliminating bugs in existing features. That was the point at which I had to cut my feature set.

When designing the feature list for your app, make sure that you have a clear vision of which features get the priority and which can be cut if necessary. For example, FallFury was originally planned to be a multiplayer game. Although it was tempting to start working on it as such from the very beginning, I knew that it wasn’t a core feature and that it wouldn’t necessarily affect the general play experience on release day. Ultimately, multiplayer capability was dropped from initial development along with a number of other not-immediately-integral features.

Expect the Unexpected

When allocating time for the project, anticipate situations that might derail the process and cause a delay in the planned delivery date. It is easy to estimate that creating an accelerometer-based motion system will take you a day or two, but you also need to allow for the potential possibility of your code not working as expected, hardware accidents, natural disasters, etc. Doing so will help you focus on the right features at the right time while diminishing the risk of missing the deadline.

Platform Samples Are the Best Documentation - Learn to Read Others’ Code

While building the core of FallFury, I leveraged multiple features that hadn’t yet become mainstream among Windows developers. Therefore, the documentation was scarce and in some instances the only help I could get was internal. Luckily, Microsoft bundled hundreds of samples that demonstrate what the new platform can do. A lot of the development approaches gained during my internship were learned from looking at the official, publicly released sample code. The lesson here: if it's not on MSDN or blogs, check the official developer samples. For example, I used the DirectX sprite drawing sample to see how to build a proper sprite rendering mechanism with behavior similar to the one available in XNA.

Atomic Commits

When I started, source control wasn’t new to me, but there were proper usage practices that I didn’t apply before the internship. One of them is atomic commits, which I learned from Clint Rutkas.

With every new addition to the application, chances are something might go wrong. It could be a faulty third-party assembly, a misplaced class, or a broken reference that brings down the entire solution. If the commits include longer time intervals, for example, you will ultimately have to roll back to a build without a lot of the new functionality. Using atomic commits, however, you’ll be able to avoid unnecessary, redundant work in the case that something goes wrong during development.

A Non-standard Approach Is Not Necessarily a Bad Approach

For loading high scores, I needed the UI layout to display the registered items. To do so, I had could either use data binding or write a simple routine that would generate XAML items on the fly in the code-behind. Data binding is a pretty common practice among XAML developers, so my initial thought was to use that, but I would have had to create a C++ binding harness in addition to the existing data retrieval code. The second approach was much cleaner because it didn’t require me to rewrite or add much code. The lesson here: pick the best approach for the job and don’t blindly follow programming trends.

Focus on the User Experience

At the end of the day, the user doesn’t care whether you wrote the application in native DirectX or XNA, whether you applied the MVVM pattern, or whether you have easily readable code. The user cares about having a great time while playing the game, regardless of its backbone. It’s your goal as a developer to deliver that user experience and ensure that the user feels comfortable with the product. This means that the development cycle goes beyond simply writing code. In FallFury, levels are dynamic, which means the source code doesn’t necessarily need to be modified in order to create a different playable experience. But that experience matters in any case, and level design is a huge part of FallFury’s final product. Users need to be immersed in the game world, so levels must be planned in such as way that the user returns to the game and doesn’t abandon it after a couple of lost rounds.

Conclusion

FallFury was my first major project that I wrote almost entirely in C++, targeting the newly released Windows 8 operating system. Although the development process was challenging, it was proven to be a great educational experience that taught me about the approaches and practices that are used internally at Microsoft.

As previously mentioned, FallFury runs on multiple types of hardware as long as that hardware supports Windows 8. This article describes the project’s general testing and debugging process, including setting the debug configurations and remote debugging.

Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-11-Hardware-Testing.  For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

Remote Debug

When working on an application that targets different machines, it’s probably out of the question to install Visual Studio on each instance and move the solution from one source to another in order to run it and diagnose potential problems. Here is where Remote Tools for Visual Studio 2012 come into play. Microsoft offers you three separate builds—tools for x86 systems, x64 systems, and ARM systems—also known as Surface RT.

Once the tools are running on the machine you want to debug, you have two choices. You can either install the remote debugger as a service, allowing it to constantly run in the background, or you can use the debugger on a per-launch basis. From a developer perspective, the choice doesn’t affect how your application is executed on the remote machine. From a security perspective, however, you need to be sure that you properly configure it so that no unwanted apps are remotely deployed.

Now you can start the Remote Debugger Configuration Wizard:

This works on both the initial start-up and also any subsequent launch as a way to easily and quickly set the necessary remote debugger settings. Specifically, it is useful to configure the machine’s network settings, allowing cross-domain communications for debugging purposes.

Once the wizard is completed, launch the Remote Debugger Monitor. Notice that it lists your machine name and the port on which it’s running. This is necessary for configuring the project to send the package to a remote machine instead of the local one:

The configuration depends on the network settings on both the local machine and the subnet as a whole. For example, in some cases, and especially on domain-joined machines, remote debugging is better done with the authentication disabled. Windows Authentication is used by default.

Since FallFury is a C++ project, the configuration for a remote session is different than, say, that of a C# Windows Store application. To configure the session, right click on the project in Solution Explorer and open the Debugging section. Make sure that Remote Machine is selected as the type of debugger to launch:

Next, specify the machine name as well as whether the current session will require authentication:

If you cannot specify the machine name, use the direct IPv4 address of that computer, minus the port (unless you’ve explicitly set it to a port other than 4016, which is the assumed default). Once the source machine connects to the remote one, you will see Visual Studio performing the deployment:

Configurations

As a matter of convenience, it is always good to have different debug configurations that will define how your projects are built, especially if the project targets multiple platforms (such as ARM and x86). Visual Studio provides a Configuration Manager that can be accessed from the debug target dropdown:

FallFury includes two separate projects—the game core, and the C#-based XML reader. Both need to be explicitly associated with separate target platforms in order to be correctly debugged. For that, profiles were created for both local and remote sessions:

As with standard Debug/Release configurations, the ones shown above determine whether the project will carry debug symbols and support debugging commands. It is important to mention that as you switch configurations, you must be careful how you configure the graphic shaders. For each shader in the container folder, explicitly set the HLSL shader type and model. Otherwise the application deployment will fail:

Remote Profiling

Last but not least, when working with different hardware configurations it might be useful to perform application profiling or performance review. Fortunately, Visual Studio also provides this capability through the vsperf tool, which is already integrated in the IDE if you are using Visual Studio 2012 Professional or above.

To initiate a profiling session on a remote device, the Remote Debugger Monitor must be active. Make sure that the Remote Machine debug target is selected, and go to Analyze > Start Performance Analysis:

On the remote machine, allow the vsperf process to run with administrative privileges. Once the profiling session completes and the data is analyzed, you can review the same performance indicators as you would when having a standard application run the profiler on the local machine:

Conclusion

Testing hardware outside the boundaries of a desktop machine is often a necessity, especially if the application relies on specific sensors, such as NFC, touch, or accelerometer. The remote debugging process is fairly streamlined and intuitive, with developing a proper network configuration allowing communication between machines requiring the most significant amount of effort. If you have problems getting the debugger to work, consult this article on MSDN.

With the release of the Windows 8 operating system, applications are now able to easily integrate with each other and have a unified way to control their workflow through unique system-wide contracts called Charms. Out of the multitude of available options, FallFury leverages two charms—Share contract and Settings. This article describes how the integration is implemented.

Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-10-Charms .  For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

The Share Contract

When a user achieves a specific score in the game, he might decide to share it with his social circle. In Charms, that’s the purpose of the Share contract, which integrates directly with the OS.

Windows Store applications have the capability to expose their sharing capabilities and register as a share target. For example, if there is a Twitter client out, it can register itself as an app through which content can be shared. FallFury, on the other hand, acts as a consumer that aggregates existing share targets and lets the user pick the one through which he wants to let the message out:

Let’s take a look at how this process is built in the code-behind. The core is located in DirectXPage.xaml.cpp—the class responsible for XAML content manipulation in FallFury. First and foremost, you need to get the current instance of the DataTransferManager class:

auto dataTransferManager = Windows::ApplicationModel::DataTransfer::DataTransferManager::GetForCurrentView();

Consider this a proxy that allows you to pass content between your app and other Windows Store apps that are executed in the context of the same sandbox. As the instance is obtained, hook it to the DataRequested event handler that will handle the scenario where the user invoked the sharing capability:

dataTransferManager->DataRequested += ref new 
    TypedEventHandler<Windows::ApplicationModel::DataTransfer::DataTransferManager^,
    Windows::ApplicationModel::DataTransfer::DataRequestedEventArgs^>(this, &DirectXPage::OnShareDataRequested);

In the OnShareDataRequested, specify the information that goes into the sharing popup:

void DirectXPage::OnShareDataRequested(Windows::ApplicationModel::DataTransfer::DataTransferManager^ manager,Windows::ApplicationModel::DataTransfer::DataRequestedEventArgs^ params)
{
    auto request = params->Request;
    request->Data->Properties->Title = "Share Score";
    request->Data->Properties->Description = "Tell your friends how much you scored in [DEN'S PROJECT]!";
    request->Data->SetText("I just scored " + StaticDataHelper::Score.ToString() + " in [DEN'S PROJECT]! Beat me! http://dennisdel.com");
}

From that point, the control is in the user’s hand. The application cannot force the share, so unless you implement a direct API hook to a social service, the Share charm will only expose the endpoints available for sharing and will let you set the shareable content. You also don’t have to worry about the way the content will be shared—that will be handled by the target application:

You can show the popup triggered by the Share charm from your application without having the user open the Charms Bar. To do this, call the ShowShareUI method:

void DirectXPage::ShareTopScore_Selected(MenuItem^ sender, Platform::String^ params)
{
    Windows::ApplicationModel::DataTransfer::DataTransferManager::ShowShareUI();
}

This is exactly what the Share button does in the screenshot above. You should make this behavior predictable.

The Settings Charm

As you just saw, integrating basic sharing capabilities in FallFury is not too complicated. Working with settings is also a fairly easy task, though it involves some XAML work. While with sharing capabilities the work focused mostly on OS-based endpoints and application-specific popups, settings allow for full control over how they’re displayed.

For all Windows Store applications, settings should be handled via the Settings charm and not through dedicated application screens. Consider which settings that directly affect the user experience might be changed in FallFury:

• Music and SFX – the user can enable or disable music and sound effects, as well as control their volume.
• Accelerometer – depending on personal preferences, the user might decide to disable the accelerometer (it is enabled by default). The accelerometer can also be inverted—if the device is tilted to the right, the character will move to the left and vice-versa. Last but not least, even with dynamic screen rotation enabled on the device, the user can disable that rotation on the application level and lock the screen to one orientation, such as portrait or landscape.
• Character Movement – the character can be easily controlled via touch (swipe) or mouse. This behavior is enabled by default, but if the user decides to only use the mouse to direct shells, he can easily disable this feature here.

As seen in the image above, the operating system provides the basic shell used to list the possible settings. Once one option is selected, however, further UI displays are delegated to the developer.

As with the share UI, the settings UI can be shown to the user from the application and not from the Charms Bar. Here is how to do this:

void DirectXPage::Settings_Selected(MenuItem^ sender, Platform::String^ params)
{
    SettingsPane::GetForCurrentView()->Show();
}

SettingsPane is the core class that handles the settings display. It does not control how settings are stored or activated. When the main page loads, you need to make sure that you hook the current SettingsPane to a CommandRequested event handler. It will be triggered when the Settings capability is invoked:

SettingsPane::GetForCurrentView()->CommandsRequested += ref new TypedEventHandler<SettingsPane^, SettingsPaneCommandsRequestedEventArgs^>(this, &DirectXPage::OnSettingsRequested);

OnSettingsRequested is the function where the core setting selections are defined and hooked to their own event handlers:

void DirectXPage::OnSettingsRequested(Windows::UI::ApplicationSettings::SettingsPane^ settingsPane, Windows::UI::ApplicationSettings::SettingsPaneCommandsRequestedEventArgs^ eventArgs)
{
    if (m_renderer->CurrentGameState == GameState::GS_PLAYING)
            StaticDataHelper::IsPaused = true;
    
    UICommandInvokedHandler^ handler = ref new UICommandInvokedHandler(this, &DirectXPage::OnSettingsSelected);
    
        SettingsCommand^ generalCommand = ref new SettingsCommand("musicSfx", "Music & SFX", handler);
        eventArgs->Request->ApplicationCommands->Append(generalCommand);
    
    SettingsCommand^ accelerometerCommand = ref new SettingsCommand("accelerometer", "Accelerometer", handler);
    eventArgs->Request->ApplicationCommands->Append(accelerometerCommand);
    
    SettingsCommand^ charMovementCommand = ref new SettingsCommand("charMovement", "Character Movement", handler);
    eventArgs->Request->ApplicationCommands->Append(charMovementCommand);
}

Each SettingsCommand is an item in the list displayed in the settings pane. When one is selected, OnSettingsSelected is called:

void DirectXPage::OnSettingsSelected(Windows::UI::Popups::IUICommand^ command)
{
    if (command->Id->ToString() == "musicSfx")
    {
        stkMusicSfx->Width = 346.0f;
        grdSubMusicSfx->Height = m_renderer->m_renderTargetSize.Height;
        stkMusicSfx->IsOpen = true;
    }
    else if (command->Id->ToString() == "accelerometer")
    {
        stkAccelerometerSettings->Width = 346.0f;
        grdAccelerometerSettings->Height = m_renderer->m_renderTargetSize.Height;
        stkAccelerometerSettings->IsOpen = true;
    }
    else if (command->Id->ToString() == "charMovement")
    {
        stkCharacterMovement->Width = 346.0f;
        grdCharacterMovement->Height = m_renderer->m_renderTargetSize.Height;
        stkCharacterMovement->IsOpen = true;
    }

    WindowActivationToken = Window::Current->Activated += ref new WindowActivatedEventHandler(this, &DirectXPage::OnWindowActivated);
}

Looking back at OnSettingsRequested, each command has a string identifier. When a command is invoked, that string identifier is returned through the IUICommand instance in the Id property. Based on that, I decided which popups to open. Since each has a similar structure, I am going to cover the implementation of just one—Music & SFX.

Here is what the end result looks like:

The panel on the left is a Popup, with two ToggleButton controls used to enable or disable generic music and sound effects. There are also two Slider controls that are used to adjust the volume. The XAML for the above layout looks like this:

<Popup HorizontalAlignment="Right" IsLightDismissEnabled="True" x:Name="stkMusicSfx" >
    <Grid Background="Black" x:Name="grdSubMusicSfx"  Width="346">
        <Grid.Transitions>
            <TransitionCollection>
                <RepositionThemeTransition />
            </TransitionCollection>
        </Grid.Transitions>

        <Grid.RowDefinitions>
            <RowDefinition Height="120"></RowDefinition>
            <RowDefinition Height="*"></RowDefinition>
        </Grid.RowDefinitions>

        <StackPanel Grid.Row="0" Orientation="Horizontal" Margin="24,12,0,0" >
            <Button Margin="0" VerticalAlignment="Center" x:Name="dismissAudioSettings" Click="dismissAudioSettings_Click" Style="{StaticResource BackButtonStyle}"></Button>
            <TextBlock Margin="12,0,0,12" Height="Auto" VerticalAlignment="Center" Text="Music &amp; SFX" Style="{StaticResource SubheaderTextStyle}"></TextBlock>
        </StackPanel>

        <StackPanel Grid.Row="1" Margin="24,24,0,0">
            <StackPanel>
                <TextBlock Text="Music" Style="{StaticResource BodyTextStyle}"></TextBlock>
                <TextBlock  Width="346" Text="This includes the theme track and level background music." Style="{StaticResource CaptionTextStyle}" TextWrapping="Wrap" Margin="0,12,12,12" ></TextBlock>
                <ToggleSwitch x:Name="tglMusic" Toggled="tglMusic_Toggled" IsOn="{Binding ElementName=XAMLPage,Path=MusicEnabled}" Margin="-6,0,0,0"></ToggleSwitch>
            </StackPanel>

            <StackPanel Margin="0,24,0,0">
                <TextBlock Text="Music Volume" Style="{StaticResource BodyTextStyle}"></TextBlock>
                <Slider x:Name="sldMusicVolume" ValueChanged="sldMusicVolume_ValueChanged" Value="{Binding ElementName=XAMLPage,Path=MusicVolume, Mode=TwoWay}" Minimum="0" Maximum="100" Margin="0,0,12,0"></Slider>
            </StackPanel>

            <StackPanel Margin="0,24,0,0">
                <TextBlock Text="Sound Effects" Style="{StaticResource BodyTextStyle}"></TextBlock>
                <TextBlock Width="346" Text="Includes sounds played during the game (e.g. explosions)." Style="{StaticResource CaptionTextStyle}" TextWrapping="Wrap" Margin="0,12,12,12" ></TextBlock>
                <ToggleSwitch x:Name="tglSFX" Toggled="tglSFX_Toggled" IsOn="{Binding ElementName=XAMLPage,Path=SFXEnabled}" Margin="-6,0,0,0"></ToggleSwitch>
            </StackPanel>

            <StackPanel Margin="0,24,0,0">
                <TextBlock Text="SFX Volume" Style="{StaticResource BodyTextStyle}"></TextBlock>
                <Slider x:Name="sldSFXVolume" ValueChanged="sldSFXVolume_ValueChanged" Value="{Binding ElementName=XAMLPage,Path=SFXVolume, Mode=TwoWay}" Minimum="0" Maximum="100" Margin="0,0,12,0"></Slider>
            </StackPanel>

        </StackPanel>

    </Grid>
</Popup>

For every Popup instance used for settings, make sure that IsLightDismissEnabled is set to true. This allows the user to dismiss the panel with a touch outside its boundaries, just like the stock system panels. Other than that, you are working with the standard XAML control set and can include virtually anything in your settings.

Notice, that the switches and sliders are bound to internal properties, such as SFXEnabled and MusicEnabled, that perform the binding via dependency property references:

DependencyProperty^ DirectXPage::_musicEnabled = DependencyProperty::Register("MusicEnabled", 
bool::typeid, DirectXPage::typeid, nullptr);
DependencyProperty^ DirectXPage::_sfxEnabled = DependencyProperty::Register("SFXEnabled", 
bool::typeid, DirectXPage::typeid, nullptr);

The properties themselves are declared in the DirectXPage header file:

static property DependencyProperty^ SFXVolumeProperty
{
    DependencyProperty^ get() { return _sfxVolume; }
}
property int SFXVolume
{
    int get() { return (int)GetValue(SFXVolumeProperty); }
    void set(int value) 
    { 
        SetValue(SFXVolumeProperty, value);
    }
}

static property DependencyProperty^ MusicVolumeProperty
{
    DependencyProperty^ get() { return _musicVolume; }
}
property int MusicVolume
{
    int get() { return (int)GetValue(MusicVolumeProperty); }
    void set(int value) 
    { 
        SetValue(MusicVolumeProperty, value);
    }
}

Let’s take a quick look at how settings are stored. I have a class called SettingsHelper that allows me to save, read, and check if specific settings exist. Here is the implementation:

#include "pch.h"
#include "SettingsHelper.h"

using namespace Windows::Storage;
using namespace Coding4Fun::FallFury::Helpers;

SettingsHelper::SettingsHelper(void)
{
}

void SettingsHelper::Save(Platform::String^ key, Platform::Object^ value)
{
    ApplicationDataContainer^ localSettings = ApplicationData::Current->LocalSettings;
    auto values = localSettings->Values;

    values->Insert(key, value);
}

Platform::Object^ SettingsHelper::Read(Platform::String^ key)
{
    ApplicationDataContainer^ localSettings = ApplicationData::Current->LocalSettings;
    auto values = localSettings->Values;

    return values->Lookup(key);
}

bool SettingsHelper::Exists(Platform::String^ key)
{
    ApplicationDataContainer^ localSettings = ApplicationData::Current->LocalSettings;
    auto values = localSettings->Values;

    return values->HasKey(key);
}

It is clear that storage and retrieval heavily relies on the ApplicationDataContainer class, the container class for local settings that eliminates the need for the developer to create his own setting files, instead delegating this task to the OS and utilizing a centralized storage for all Windows Store applications.

A typical scenario that utilizes the class above is executed when the toggle that manages the sound effects is switched:

void DirectXPage::tglSFX_Toggled(Platform::Object^ sender, Windows::UI::Xaml::RoutedEventArgs^ e)
{
    SettingsHelper::Save("sfxEnabled", tglSFX->IsOn);
    SFXEnabled = tglSFX->IsOn;
    AudioManager::IsSFXStarted = SFXEnabled;

    if (SFXEnabled)
    {
        AudioManager::AudioEngineInstance.StartSFX();
    }
    else
    {
        AudioManager::AudioEngineInstance.SuspendSFX();
    }
}

The Boolean value above will be automatically serialized and stored. The files will be located at C:\Users\YOUR_USER_NAME\AppData\Local\Packages\PACKAGE_ID\Settings\settings.dat:

Conclusion

Implementing sharing through the OS channel in Windows 8 is extremely easy seeing as the developer does not necessarily have to worry about connecting the app to third-party API endpoints. Instead, the user controls the sharing, allowing flexibility of choice without requiring a major addition to the existing code base. It’s hard to predict which services might appear, and modifying the app to support each one of them would be next to impossible otherwise.

You can read more about settings in Windows Store applications here.

During gameplay there are scenarios during which users need to be visually notified that something has happened, such as a collision with an obstacle or an enemy shell. One way to do this is by having explosion or item breaking simulation, which brings us to the next large component in FallFury—the sprite-based particle system. It doesn’t offer as much power as a full-fledged particle system would, but it allows for effects that fit well within the overall theme and layout of the game.

Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-9-Particle-System .  For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

The Core

ParticleSystem is the dedicated folder in the project that contains everything needed to render multiple textures at once and displace them to create the desired effect:

A single particle carries information regarding its size, position, velocity, color shading, rotation, circular velocity, and scale. As with any other rendered entity, it has a bounding box that can be used to detect its intersection with other elements on the screen. In FallFury, this functionality is not used.

Its structure is as follows:

#pragma once
#include "pch.h"
struct Particle
{
    Particle(float2 size);
    Particle(float2 size, float4 shading);
    float2 Size;
    float2 Position;
    float2 Velocity;
    float4 Shading;
    float Rotation;
    float RotationVelocity;
    float Scale;
    bool IsWithinScreenBoundaries(float x, float y, Windows::Foundation::Rect screenBounds);
    Windows::Foundation::Rect GetBoundingBox();
};

The Particle class also happens to have two constructors—one that sets the particle to have the default shading, effectively removing the effect, and one where shading is dynamic. Note that a particle on its own doesn’t do much—it neither carries the associated texture nor has an internal loop that can be used in any given application part to display it.

The next core class is ParticleSet. It is used as a container for all the particles associated with a specific effect. For example, if I want to create flying stuffing when the bear hits an obstacle, I create a new ParticleSet instance and define the necessary particle properties:

• Lifespan – a particle set does not constantly animate. It displays the particles for a limited amount of time, and displaces them by the given velocity values, and then self-destructs.
• Texture – all particles in a ParticleSet have the same texture. Going back to the stuffing example, there is a single PNG file used to render multiple variable-sized particles on collision.
• IsAlive – this is the flag that shows whether the particle set should be rendered in the first place. If it is set to false then, regardless of the conditions, this ParticleSet instance is ignored.
• ShouldScale – this flag determines whether particles will automatically increase their scale as they are being displaced, creating the effect of a particle approaching the screen. This effect is applied to each particle in the set.

The container class is used to update the particles through the Update function:

void ParticleSet::Update(float timeDelta)
{
    float quat = _lifespan / 0.016f;
    float decrement = 1.0f / quat;
    
    if (_totalTime <= _lifespan && _isAlive)
    {
        _totalTime += timeDelta;
        
        for (auto particle = _particles.begin(); particle != _particles.end(); particle++)
        {
            if (_shouldScale)
                particle->Scale += 0.2f;
            
            particle->Shading.a -= decrement;
            particle->Position = float2(particle->Position.x + particle->Velocity.x,
                            particle->Position.y + particle->Velocity.y);
            
            if (!_shouldScale)
                particle->Rotation += particle->RotationVelocity;
        }
    }
    else
    {
        _totalTime = 0.0f;
        _isAlive = false;
    }
}

As it relies on the _isAlive flag, the Update loop is only used when the particle displacement is activated via the Activate function:

void ParticleSet::Activate(float2 position, float2 velocity, bool randomize, bool scale)
{
    for (auto particle = _particles.begin(); particle != _particles.end(); particle++)
    {
        particle->Position = position;

        if (randomize)
            particle->Velocity = float2(RandFloat(-5.0f,5.0f), RandFloat(-5.0f, 5.0f));
        else
            particle->Velocity = float2(velocity.x + RandFloat(-0.6f, 0.6f), velocity.y + RandFloat(-0.6f, 0.6f));;
    }

    _shouldScale = scale;
    _isAlive = true;
}

When a set is activated, several user-defined parameters come into play. The position is set no matter what and is used to create the source point from which the particles start appearing. The velocity, on the other hand, can be randomized between the values of -5 and 5 pixels per update loop, on both the X-axis and the Y-axis. If randomized, large amount of particles will create an explosion that starts from the center point and expands towards all quadrants. When the velocity is not randomized, it creates a triangular expansion grid on which particles deviate from the center point in one of the given directions:

When it’s time to render the particles, the sprite batch associated with the current game screen is used to pass a texture for each particle registered in the set:

void ParticleSet::Render(SpriteBatch ^spriteBatch)
{
    for (auto particle = _particles.begin(); particle != _particles.end(); particle++)
        {
        if (GamePlayScreen::Manager->IsWithinScreenBoundaries(particle->Position))
                    spriteBatch->Draw(_texture.Get(), particle->Position, PositionUnits::DIPs,
                    particle->Size * particle->Scale, SizeUnits::Pixels, particle->Shading, particle->Rotation);
        }
}

Although there is no flag check inside the Render method that would make sure that the set is alive, this can be done outside of it by calling IsAlive, which will return the flag value:

bool ParticleSet::IsAlive()
{
    return _isAlive;
}

Let’s now take a look at the class that managed the particle flow—ParticleCore.

The Particle Manager

ParticleCore is the class that manages internal particle sets, and is also the proxy for set activation, update, and rendering. Here is its structure:

#pragma once
#include "pch.h"
#include "ParticleSet.h"
#include <list>
#include <map>

class ParticleCore
{
    public:
    ParticleCore();
    ParticleCore(Coding4Fun::FallFury::Screens::GameScreenBase^);
    virtual ~ParticleCore();
    
    void CreatePreCachedParticleSets();
    void ActivateSet(Platform::String^, float2);
    void ActivateSet(Platform::String^, float2, float2);
    void ActivateSet(Platform::String^, float2, bool);
    void ActivateSet(Platform::String^, float2, float2, bool);
    void ActivateSet(Platform::String^, float2, float2, bool, bool scale);
    void Update(float);
    void Render();
    
    private:
        std::list<ParticleSet>                                                    _renderParticleSets;
        std::map<Platform::String^, ParticleSet*>                                _particleSetCache;
        std::map<Platform::String^, Microsoft::WRL::ComPtr<ID3D11Texture2D>>    _textureCache;
        Coding4Fun::FallFury::Screens::GameScreenBase^                            _screenBase;
};

As previously mentioned, set activation can be done with some omitted parameters inferred by the system, such as randomization of velocity or particle scaling, which is the reason why you see multiple overloads for ActivateSet.

ParticleCore is also the container for pre-defined sets that have specific textures and properties that are dumped in the particle set cache (internal _particleSetCache). The cache is reusable and even though sets can be activated and destroyed, the cache remains intact for the duration of the game unless explicitly reset or modified. The texture cache is an addition to the particle set cache and is used as a helper container to temporarily store the textures used for individual particles.

Taking a look under the hood at the CreatePreCachedParticleSets function, you can see multiple sets that can be used in the game, each with different properties. Here is a snippet that shows how the bear stuffing explosion set is created:

auto smallExplosionSet = new ParticleSet(_textureCache["Stuffing"], LIFESPAN);
for (int i = 0; i < 20; i++)
{
    float size = RandFloat(50.0f, 100.0f);
    Particle particle(float2(size, size));
    smallExplosionSet->AddParticle(particle);
}

Once all the particles are in place for that specific set, it is added to the global particle set cache:

_particleSetCache["SmallExplosion"] = smallExplosionSet;

The particle set cache is not explicitly used to render anything on the screen. Rather, that task is delegated to the rendering cache. When a set is activated, the cache is inspected for the given key, its Activate function is called, marking it as alive, and the set itself is pushed on the rendering stack:

void ParticleCore::ActivateSet(Platform::String^ name, float2 position, float2 velocity, bool randomize, bool scale)
{
    ParticleSet set = ParticleSet(*_particleSetCache[name]);
    set.Activate(position, velocity, randomize, scale);
    _renderParticleSets.push_back(set);
}

The Render loop takes care of invoking the proper SpriteBatch drawing functions for each set that is in that stack:

void ParticleCore::Render()
{
    for (auto set = _renderParticleSets.begin(); set != _renderParticleSets.end(); ++set)
    {
        if ((*set).IsAlive())
        {
            (*set).Render(_screenBase->CurrentSpriteBatch);
        }
    }
}

Notice that, as previously mentioned, the set does not perform the life check on itself when rendering. Instead, the manager class performs this action. The same applies to the Update loop:

void ParticleCore::Update(float timeDelta)
{
    for (auto set = _renderParticleSets.begin(); set != _renderParticleSets.end();)
    {
        if ((*set).IsAlive())
        {
            (*set).Update(timeDelta);
            ++set;
        }
        else
        {
            set = _renderParticleSets.erase(set);
        }
    }
}

The textures for each set are individually loaded in the CreatePreCachedParticleSets. Each instance is internally pushed into the cache and also added to the sprite batch associated with the current game screen, where ParticleCore is used:

Loader->LoadTexture("DDS\\stuffing.dds", &_textureCache["Stuffing"], nullptr);
_screenBase->CurrentSpriteBatch->AddTexture(_textureCache["Stuffing"].Get());

Once everything is loaded, the ParticleCore is ready to go and you can use as many particles as necessary in any part of the application. In GamePlayScreen, particle sets are activated in many cases. For example, if the bear dies:

void GamePlayScreen::CheckBearHealth()
{
    if (GameBear->CurrentHealth <= 0)
    {
        m_particleSystem.ActivateSet("Buttons",GameBear->Position, true);
        GameBear->Kill();
        StopBackground();
    }
}

Conclusion

Implementing a sprite-based particle system is not complicated, but it requires depending on a number of assumptions. For example, when a particle set is created, you might consider the fact that some hardware can handle drawing only a given number of sprites at the same time. If a particle set is rendered on a desktop machine, there is no guarantee that the same set will successfully render on an ARM device. Therefore, plan accordingly. For each particle set, create a lifespan that fits the scenario without wasting resources on rendering unnecessary particles. As an additional failsafe, you might want to disable specific particle set types when a Direct3D feature level below 10.0 is detected.

During the gameplay multiple entities interact with each other to make the gaming experience what it is. The bear collides with obstacles and collects buttons, monsters shoot shells that can fly off-screen or hit the bear—all this is possible with the help of the basic collision detection techniques that are implemented in FallFury.

Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-8-Element-Interaction . For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

Buttons

Buttons are bonus-boosters that can be placed by the level designer anywhere on the screen in game mode. These are relatively small entities, which are displaced vertically with each cycle of the Update loop and move in the opposite direction, but with the same velocity, as the main character.

Looking at the Update function in the GamePlayScreen class, you will notice this call:

UpdateButtons();

UpdateButtons can be considered the button manager function responsible for removing the collected buttons from the rendering stack, counting them, and checking for a button collision when the bear is in close proximity. The implementation looks like this:

void GamePlayScreen::UpdateButtons()
{
    Windows::Foundation::Rect livingEntityBoundingBox = GameBear->GetBoundingBox();
    for (auto button = m_buttons.begin(); button != m_buttons.end();)
    {
        (*button)->Position.x = (*button)->PixelDiff + LoBoundX;
        (*button)->Position.y -= GameBear->Velocity.y;
        if (Geometry::IsInProximity(GameBear->Position,(*button)->Position, 100))
        {
            Windows::Foundation::Rect obstacleRect = (*button)->GetBoundingBox();
            if (livingEntityBoundingBox.IntersectsWith(obstacleRect))
            {
                AudioManager::AudioEngineInstance.StopSoundEffect(Coin);
                AudioManager::AudioEngineInstance.PlaySoundEffect(Coin);
                m_particleSystem.ActivateSet("Sparkle", (*button)->Position,float2(RandFloat(-6.0f,6.0f),RandFloat(-10.0f, -5.0f)));
                StaticDataHelper::ButtonsCollected++;
                button = m_buttons.erase(button);
            }
            else
            ++button;
        }
        else
        {
            ++button;
        }
    }
}

For performance reasons, FallFury supports composite bounding box creation as well as simple box creation. As mentioned earlier in the series, the main character is not composed of a single texture, but rather multiple sprites that are cross-positioned to create a single visual entity. To give you a better idea of what composite vs. simple boxing looks like, take a look at the images below:

The image on the left shows how each part of the bear has its own bounding box, and each will be used for collision checking. In the image on the right, the bear has a single bounding box, creating minor potential gaps, but gaining performance.

Going back to UpdateButtons, once the bounding box is obtained, I iterate through the button collection and make sure that each item is located in the proper space:

(*button)->Position.x = (*button)->PixelDiff + LoBoundX;
(*button)->Position.y -= GameBear->Velocity.y;

The constant position checks are necessary because FallFury supports dynamic orientation changes. When the user switches from portrait to landscape mode and vice-versa, rendered elements on the screen are not automatically repositioned. Setting the X position is easy as long as there is a fixed button margin (from the left side of the screen: LoBoundX) and adding it to the current LoBoundX value results in a proper X location. There is no need to do the same check on the Y-axis because the level length remains the same regardless of the current screen orientation. The adjustment made relative the Y position is bound to the bear velocity. If the bear moves slower, buttons will also scroll slower.

Given that all buttons are properly positioned, a proximity check is performed on each button passed through the loop. If the bear position is at least 100 pixels away from the current button, the corresponding bounding box is obtained and an intersect check is performed. In simple boxing mode, this is done via Windows::Foundation::Rect::IntersectsWith:

If a collision occurs, the appropriate sound effect is played and a particle set is activated to create a visual notification of the action. After the button counter is incremented, the button is removed from the local collection, effectively being removed from the rendering stack.

Power-ups

As the bear flies towards the end of the level, it might encounter bonuses to improve its ability to fight incoming enemies or protect from damage caused by enemy ammo or obstacles. The process behind displaying power-ups on the screen and determining whether there was a collision with the main character is similar to UpdateButtons.

The core function for this task is UpdatePowerups:

void GamePlayScreen::UpdatePowerups(float timeDelta)
{
    if (m_powerups.size() > 0)
    {
        for (auto powerup = m_powerups.begin(); powerup != m_powerups.end(); powerup++)
        {
            (*powerup)->Update(timeDelta);
            (*powerup)->Position.x = (*powerup)->PixelDiff + LoBoundX;
            (*powerup)->Position.y -= GameBear->Velocity.y;
        }
    }
    CheckForCollisionWithPowerups();
}

One difference you probably noticed in the snippet above is the fact that the collision check is now done through a separate function—CheckForCollisionWithPowerups:

void GamePlayScreen::CheckForCollisionWithPowerups()
{
    Powerup^ currentPowerup;
    Windows::Foundation::Rect livingEntityBoundingBox = GameBear->GetBoundingBox();
    for (auto powerup = m_powerups.begin(); powerup != m_powerups.end();)
    {
        currentPowerup = (*powerup);
        if (Geometry::IsInProximity(GameBear->Position,currentPowerup->Position, 100))
        {
            Windows::Foundation::Rect obstacleRect = currentPowerup->GetBoundingBox();
            if (livingEntityBoundingBox.IntersectsWith(obstacleRect))
            {
                AudioManager::AudioEngineInstance.PlaySoundEffect(GenericPowerup);
                GameBear->PickupPowerup(currentPowerup, &m_particleSystem);
                powerup = m_powerups.erase(powerup);
            }
            else
            ++powerup;
        }
        else
        {
            ++powerup;
        }
    }
}

If an intersection is detected between the boxed bear and the power-up texture, the current power-up is passed to the Bear instance and the appropriate type of action is selected:

void Bear::PickupPowerup(Powerup^ powerup, ParticleCore* ParticleSystem)
{
    switch (powerup->Type)
    {
        case PowerupType::PARACHUTE:
        {
            m_previousVelocity = Velocity.y;
            SetParachute(powerup->Lifespan);
            Velocity.y -= powerup->Effect;
            ParticleSystem->ActivateSet("ScalableParachute", Position, 0.0f, false, true);
            break;
        }
        case PowerupType::HEALTH:
        {
            CurrentHealth = MaxHealth;
            ParticleSystem->ActivateSet("ScalableHeart", Position, 0.0f, false, true);
            break;
        }
        case PowerupType::BUBBLE:
        {
            if (IsHelmetEnabled)
            {
                IsHelmetEnabled = false;
                DamageDivider = 1.0f;
            }
            else
            {
                DamageDivider = powerup->Effect;
            }
            IsBubbleEnabled = true;
            m_maxBubbleCounter = powerup->Lifespan;
            ParticleSystem->ActivateSet("ScalableBubble", Position, 0.0f, false, true);
            break;
        }
        // [...]
        case PowerupType::BOOMERANG:
        {
            m_weaponType = powerup->Type;
            m_weaponTexture = m_boomerangTexture;
            CurrentDamage = powerup->Effect;
            m_weaponSize = float2(225.0f, 205.0f) * 0.5f;
            ParticleSystem->ActivateSet("ScalableBoomerang", Position, 0.0f, false, true);
            break;
        }
    }
}

Depending on the power-up, textures are added to the bear model and later passed to the rendering stack (if the power-up type is PARACHUTE), some textures and capabilities are replaced (BOOMERANG), or the bear capabilities are temporarily modified (BUBBLE):

If one of the temporary power-ups is enabled n the Update loop, dedicated timers ensure that ability enhancement does not last longer than necessary. As an example, here is the snippet that controls the bubble:

if (IsBubbleEnabled)
{
    m_currentBubbleCounter += timeDelta;
    if (m_currentBubbleCounter > m_maxBubbleCounter)
    {
        IsBubbleEnabled = false;
        DamageDivider = 1.0f;
        m_currentBubbleCounter = 0.0f;
    }
}

Ammo Collisions

There are also objects that direct ammo at either the enemy or the bear. After release, each shell follows a linear path towards the target, and while the user directs the shells that originate from the bear, those released by enemy entities automatically target the bear. In this case, the ammo needs to collide with an object to either damage or kill it. When a shell is released, it is added to the general ammo collection that is later updated internally:

void GamePlayScreen::UpdateAmmo(float timeDelta)
{
    for (auto shell = m_ammoCollection.begin(); shell != m_ammoCollection.end(); shell++)
    {
        (*shell)->Update(timeDelta, &m_particleSystem);
    }
}

At this point, the rendering system does not differentiate between friendly and enemy ammo—all it knows is that each item in the collection must have a new position when a new frame is rendered. The CheckForCollisionsWithAmmo function checks for ammo collisions:

void GamePlayScreen::CheckForCollisionWithAmmo(LivingEntity^ entity)
{
    if (entity != nullptr)
    {
        Windows::Foundation::Rect livingEntityBoundingBox = entity->GetBoundingBox();
        if (entity->IsFriendly)
        {
            for (auto ammo = m_ammoCollection.begin(); ammo != m_ammoCollection.end();)
            {
                if (!(*ammo)->IsFriendly)
                {
                    Windows::Foundation::Rect ammoBoundingBox = (*ammo)->GetBoundingBox();
                    if (livingEntityBoundingBox.IntersectsWith(ammoBoundingBox))
                    {
                        m_particleSystem.ActivateSet("SmallExplosion",entity->Position, true);
                        entity->InflictDamage((*ammo)->HealthDamage);
                        GameBear->RedShade();
                        AudioManager::AudioEngineInstance.PlaySoundEffect(OuchA);
                        AudioManager::AudioEngineInstance.PlaySoundEffect(HardSoftCollision);
                        CheckBearHealth();
                        ammo = m_ammoCollection.erase(ammo);
                    }
                    else
                    {
                        ++ammo;
                    }
                }
                else
                {
                    ++ammo;
                }
            }
        }
        else
        {
            for (auto ammo = m_ammoCollection.begin(); ammo != m_ammoCollection.end();)
            {
                if ((*ammo)->IsFriendly)
                {
                    Windows::Foundation::Rect ammoBoundingBox = (*ammo)->GetBoundingBox();
                    if (livingEntityBoundingBox.IntersectsWith(ammoBoundingBox))
                    { 
                        Monster^ monster = ((Monster^)entity);
                        if (!monster->IsDead && monster->IsActive)
                        {
                            m_particleSystem.ActivateSet("SmallExplosion", entity->Position, true);
                            entity->InflictDamage((*ammo)->HealthDamage);
                            monster->RedShade();
                            monster->CheckIfAlive();
                            AudioManager::AudioEngineInstance.PlaySoundEffect(SharpSoftCollision);
                        }
                        ammo = m_ammoCollection.erase(ammo);
                    }
                    else
                    {
                        ++ammo;
                    }
                }
                else
                {
                    ++ammo;
                }
            }
        }
    }
}

When the function is called, it is usually run against an entity that is present on the screen, such as the main character. Regardless of whether the enemy or the friendly character fired the shot, the shot cannot inflict damage to its source, and that’s why the function implements the ammo-to-entity crosscheck. If the ammo collides with any shells intersecting the entity bounding box, a collision is counted and health verification is performed to ensure that the character is still alive and that the game should continue. The bear’s health is checked via CheckBarHealth:

void GamePlayScreen::CheckBearHealth()
{
    if (GameBear->CurrentHealth <= 0)
    {
        m_particleSystem.ActivateSet("Buttons",GameBear->Position, true);
        GameBear->Kill();
        StopBackground();
    }
}

That said, not all ammo will collide with entities on the screen. Some of it will go out-of-bounds, and without an explicit cleanup process in place out-of-bounds ammo is constantly re-rendered even though the end user has no way of seeing it. To avoid this, there is a helper function—CheckForOutOfBoundsAmmo:

void GamePlayScreen::CheckForOutOfBoundsAmmo()
{
    for (auto l_Iter = m_ammoCollection.begin(); l_Iter != m_ammoCollection.end(); /* nothing here */ )
    {
        if (!GamePlayScreen::Manager->IsWithinScreenBoundaries((*l_Iter)->Position))
        {
            l_Iter = m_ammoCollection.erase(l_Iter);
        }
        else
        {
            ++l_Iter;
        }
    }
}

If any shell flies outside the screen bounding box, its instance is erased from the collection and the renderer no longer worries about allocating memory for an irrelevant item. To give you an idea of how that happens, here is a snippet that shows how the RenderScreen function handles the current ammo set:

if (!m_ammoCollection.empty())
{
    for (auto shell = m_ammoCollection.begin(); shell != m_ammoCollection.end(); shell++)
    {
        if (!(*shell)->IsFriendly)
        {
            (*shell)->Render();
        }
        else
        {
            GameBear->RenderShell((*shell)->Position, (*shell)->Rotation);
        }
    }
}

Conclusion

Element interaction is a core part of the FallFury experience. Separate handlers are implemented for each of them to ensure maximum flexibility when it comes to adding or removing components without breaking major parts of the code-base. Handling is mainly accomplished in the Update loop by iterating through registered entity sets, such as ammo, and verifying whether an action should be taken. Be cautious when implementing this kind of scenario with large entities and data sets—having multiple loops running simultaneously might tax machine performance, especially on low-power configurations such as ARM.

FallFury relies on a lot of dynamic content. As you already aware of how SpriteBatch is invoked inside the FallFury rendering stack, this article focuses on how dynamic activities are handled on existing textures and entities. If you need a quick look at what was already covered, refer to Part 3 of the series.

Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-7-Animations . For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

Menu Screen

The first animated element shown in the menu screen is the bear, which is positioned in the top half of the viewport. Notice that the bear moves its paws as well as moving across the screen:

The bear is implemented in the GameBear class and is the normal playable entity that is invincible when animated outside the scope of the gameplay screen. Its full body image is composed of four textures:

Microsoft::WRL::ComPtr<ID3D11Texture2D>        m_head;
Microsoft::WRL::ComPtr<ID3D11Texture2D>        m_leftPaw;
Microsoft::WRL::ComPtr<ID3D11Texture2D>        m_rightPaw;

Microsoft::WRL::ComPtr<ID3D11Texture2D>        m_body // Exposed through LivingEntity;

Each of these textures is internally dependent on the coupled float rotation and position values passed to the sprite batch in the host container (the parent screen):

HostContainer->CurrentSpriteBatch->Draw(
    m_rightPaw.Get(),
    m_rightPawPosition,
    PositionUnits::DIPs,
    float2(522.0f, 141.0f) * Scale,
    SizeUnits::Pixels,
    m_shading,
    m_rightPawRotation - 0.5f);

HostContainer->CurrentSpriteBatch->Draw(
    m_body.Get(),
    Position,
    PositionUnits::DIPs,
    float2(400.0f, 400.0f) * Scale,
    SizeUnits::Pixels,
    m_shading,
    Rotation);

HostContainer->CurrentSpriteBatch->Draw(
    m_leftPaw.Get(),
    m_leftPawPosition,
    PositionUnits::DIPs,
    float2(522.0f, 141.0f) * Scale,
    SizeUnits::Pixels,
    m_shading,
    m_leftPawRotation + 0.5f);

Let’s take a look at how the position and rotation values are modified. All item animations are performed with the help of timers set to brief intervals in which the values are cycled through two value exchange operations. Here is a snippet taken from inside the Update method in the GameBear class that is responsible for bear arm movement:

if (m_armRotationTimer < 0.6f)
{
    m_rightPawRotation -= 0.005f;
    m_leftPawRotation += 0.005f;
}
else if (m_armRotationTimer > 0.6f && m_armRotationTimer < 1.2f)
{
    m_rightPawRotation += 0.005f;
    m_leftPawRotation -= 0.005f;
}
else
{
    m_rightPawRotation = 0.0f;
    m_leftPawRotation = 0.0f;
    m_armRotationTimer = 0.0f;
}

m_armRotationTimer is a float value that is initially set to 0.0f. In the Update loop and increments each iteration by the value of timeDelta, which represents the time difference between two loop cycles. As this value is less than 600 milliseconds (0.6 seconds), the right paw rotation is incremented and the left paw rotation decremented. The difference in the value adjustment direction is caused by the fact that one paw is rotated clockwise and the other is rotated counterclockwise. Once the timer tracks a value above 600 milliseconds but below than 1200 milliseconds, the process reverses and the paws rotate in the opposite direction. After 1.2 seconds, the timer resets, as does the rotation.

Since this rotation is automated, GameBear->Update is called in the menu screen:

m_showBear->Update(timeTotal, timeDelta, float2(Manager->m_windowBounds.Width / 2, Manager->m_windowBounds.Height / 2));

Outside the context of the bear update cycle, there is a similar timed mechanism used to randomly move the bear across the screen:

if (m_positionYAdj == 0.0f)
{
    m_positionYAdj = RandFloat(0.1f, 1.2f);
    m_positionXAdj = RandFloat(-1.0f, 1.0f);
}

if (m_positionTimer < 4.0f)
{
    m_showBear->Position.x += m_positionXAdj;
    m_showBear->Position.y += m_positionYAdj;
    m_showBear->Rotation += 0.001f;
    m_showBear->Scale += 0.0008f;
    
    m_showMonster->Scale += 0.002f;
}
else if (m_positionTimer > 4.0f && m_positionTimer < 8.0f)
{
    m_showBear->Position.x -= m_positionXAdj;
    m_showBear->Position.y -= m_positionYAdj;
    m_showBear->Rotation -= 0.001f;
    m_showBear->Scale -= 0.0008f;
}
else
{
    m_positionYAdj = 0.0f;
    m_positionTimer = 0.0f;
}

The X and Y adjustments are randomly generated and for the duration of a single timer cycle (in this case, 4 seconds) the bear’s horizontal and vertical positions are adjusted using those values. The rotation and scale are minimally adjusted to create a 3D motion effect. Because the displacement is minimal, the bear is always visible and does not move much outside the viewport.

There is also a flying monster animation in the main menu screen. Monster creation happens on top of the same instance that is replaced when an initial monster flies out of bounds:

m_showMonster->Position.y -= m_showMonster->Velocity.y;
if (m_showMonster->Position.y < -m_showMonster->Size.y)
{
    CreateNewMonster();
}

When a new monster is created, a random monster type is selected and its position on the X axis is randomized. The Y position is set to be right under the bottom screen boundary:

void MenuScreen::CreateNewMonster()
{
    m_showMonster = ref new Monster(this, (MonsterType)(rand() % (int)MonsterType::MT_CANDYLAND_E), true);
    m_showMonster->Scale = 0.3f;
    m_showMonster->Velocity.y = 1.0f;
    m_showMonster->Load();
    m_showMonster->Position = float2(RandFloat(LoBoundX, HiBoundX), Manager->m_windowBounds.Height + m_showMonster->Size.y * m_showMonster->Scale);
}

Monster scaling is done on the same timed loop as the bear. The results look like this:

Gameplay Screen

Bear

The gameplay screen carries the most animations in FallFury, including animations related to ammo displacement, monster movement paths, power-up animations, and secondary game character animations that are triggered in special cases such as character death.

Let’s start with character entrance. When the game begins, the teddy bear falls into the screen and stops at a specific point close to the top boundary of the gameplay screen. Once that transition is complete, the bear is no longer vertically displaced:

if ((GameBear->Position.y / HiBoundY) < 0.19f)
{
    GameBear->Position.y += GameBear->Velocity.y * (3.2f);
}

If the limit is not hit, the original bear velocity is multiplied by a fixed value, after which the condition is ignored for the rest of the gameplay.

Let’s now take a look at what happens when an enemy shell kills the bear. Generally, when the main character is killed, there is not much sense in maintaining other in-game activities. In FallFury, a killed bear results in a time freeze—the fall stops and the monsters are no longer shooting or moving. Not only that, but the bear is also flipped on its back:

In the Update loop, the position for the paws and the head are adjusted for the new body texture layout:

if (IsDead)
{
    if (m_armRotationTimer < 1.0f)
    {
        m_rightPawRotation += 0.001f;
        m_leftPawRotation -= 0.001f;
    }
    else if (m_armRotationTimer > 1.0f && m_armRotationTimer < 2.0f)
    {
        m_rightPawRotation -= 0.001f;
        m_leftPawRotation += 0.001f;
    }
    else
    {
        m_rightPawRotation = 0.0f;
        m_leftPawRotation = 0.0f;
        m_armRotationTimer = 0.0f;
    }

    m_rightPawPosition = Position + float2(40.0f, -50.0f) * Scale;
    m_leftPawPosition = Position + float2(10.0f, 150.0f) * Scale;
    m_headPosition = (Position - float2(-160.0f, 0.0f) * Scale);
}

The proper textures are assigned via the Kill method:

void Bear::Kill()
{
    IsDead = true;
    Rotation = -1.0f;
    m_head = m_deadHead;
    m_body = m_deadBody;
    m_leftPaw = m_deadLeftArm;
    m_rightPaw = m_deadRightArm;
}

The bear will continue falling as long as StopBackground in GamePlayScreen is called:

void GamePlayScreen::StopBackground()
{
    m_isBackgroundMoving = false;
}

This will cause the internal state check to fail in the Update loop, sending the bear off screen limits. Once the bear completes the fall, the state is set to GS_GAME_OVER:

if (!m_isBackgroundMoving)
{
    if (GameBear->Position.y > m_screenSize.y)
    {
        Manager->CurrentGameState = GameState::GS_GAME_OVER;
    }
    else
    {
        GameBear->Position.y += GameBear->Velocity.y * 1.5f;
    }
}

Monsters

Monsters, on the other hand, are being constantly moved during the duration of the game. Their basic displacement is performed by MoveMonsters:

void GamePlayScreen::MoveMonsters(float timeTotal ,float timeDelta)
{
    for (auto monster = m_monsters.begin(); monster != m_monsters.end();)
    {
        Monster^ currentMonster = (*monster);
        if ((currentMonster->Position.y - GameBear->Position.y) < -Manager->m_renderTargetSize.Height / 2)
        {
            monster = m_monsters.erase(monster);
        }
        else
        {
            currentMonster->Velocity.y = GameBear->Velocity.y;
            currentMonster->Update(timeTotal, timeDelta, GameBear->Position, GetScreenBounds());
            CheckForCollisionWithAmmo(currentMonster);
            ++monster;
        }
    }
}

This method adjusts the monster position relative to the bear:

The zig-zag like motion is defined in the Monster class in the Update method:

float adjustment = 0.0f;
adjustment = m_goingRight ? Position.x + Velocity.x : Position.x - Velocity.x;
if (adjustment >= (HostContainer->LoBoundX + (Size.x * Scale) / 2.0f) 
&& adjustment <= (HostContainer->HiBoundX - (Size.x * Scale) / 2.0f))
{
    Position.x = adjustment;
}
else
{
    m_goingRight = !m_goingRight;
}

The snippet above determines the direction in which the enemy has to move depending on which screen boundary is hit first. While the monster is active and is visible, its vertical adjustment is performed in a timed loop, like this:

m_jumpingTimer += timeDelta;
if (m_jumpingTimer > 0.0f && m_jumpingTimer < 1.0f)
{
    Position.y -= 1.0f;
}
else if (m_jumpingTimer >= 1.0f && m_jumpingTimer < 2.0f)
{
    Position.y += 1.0f;
}
else
{
    m_jumpingTimer = 0.0f;
}

Here you can either use the Y velocity or introduce a static value. The only condition has to be a number low enough that the monster does not leave the screen, which would prevent the bear from being able to kill it. When the monster is killed, it flies out by following an arch-like path:

The “death arc” is implemented via another timer:

Position.x += Velocity.y * 1.3f;
if (m_deathArcTimer > 0.4f)
{
    Position.y += Velocity.y;
    if (Scale > 0.1f)
    Scale -= 0.01f;
}
else
{
    Position.y -= Velocity.y;
    Scale += 0.01f;
    m_deathArcTimer += timeDelta;
}

Regardless of the monster position, the horizontal displacement is positive and the entity moves to the right. For 400 milliseconds its vertical position is decreased, moving the texture up, and the scale is also adjusted to create the proximity effect. After this time interval, the monster drops out of the screen boundaries.

As previously mentioned, each monster is composed of three cycled textures, which are loaded and stored in three ID3D11Texture2D containers:

Microsoft::WRL::ComPtr<ID3D11Texture2D> m_spriteA;
Microsoft::WRL::ComPtr<ID3D11Texture2D> m_spriteB;
Microsoft::WRL::ComPtr<ID3D11Texture2D> m_spriteC;

Each of these is assigned to the main body texture container every 300 milliseconds, replacing the previously assigned asset:

m_stateChangeTimer += timeDelta;
if (m_stateChangeTimer < 0.3f)
{
    m_body = m_spriteA;
}
else if (m_stateChangeTimer > 0.3f && m_stateChangeTimer < 0.6f)
{
    m_body = m_spriteB;
}
else if (m_stateChangeTimer > 0.6f && m_stateChangeTimer < 0.9f)
{
    m_body = m_spriteC;
}
else if (m_stateChangeTimer > 0.9f)
{
    m_stateChangeTimer = 0.0f;
}

This cycle can be disabled when the monster is not active.

Ammo

There are different types of ammo used in the game, and each behaves in its own way. The stock ammo type is a plasma ball.

The default shooting behavior releases a shell when the user taps anywhere on the screen (other than the pause button area). Given the texture of the shell, when it is released the tail must be oriented towards the bear at an angle equal to the one created by the bear and the target position. To get a better idea, take a look at the images below:

In the image on the left, the player tapped at the bottom of the screen. In the one on the right, the player tapped at the right edge of the screen. The shell accordingly rotates depending on the tapped position. To get a better understanding of how the rotation is performed, imagine the XY coordinate grid with the bear located at the intersection of the axes:

Thinking back to trigonometry, notice the triangle formed by the shell and the bear origin point:

Remember also the trig concept of quadrants:

The quadrant in which the shell is located determines how the angle is calculated and the tail rotated. If the shell is in quadrant 2 or 3, calculate the rotation radians by following this algorithm:

1. Find the hypotenuse from the X and Y components of the velocity, which can be calculated by finding the difference between the bear and the position of the tap. To do this, us the well-known Pythagorean theorem:

The hypotenuse formula can be deduced from the formula above and is the square root of the sum of the velocity component squares:

2. Once calculated, it is necessary to find the sine of the target angle. The value can be obtained by dividing the Y component of the velocity by the hypotenuse.

3. Find the angle by calculating the arcsine from the resultant sine value. This will return the final angle in radians.

That said, for shells launched in quadrants 1 or 4, there is one extra step that needs to be performed. In addition to the rotation value obtained from the steps listed above, the rotation should be incremented by the radian value of a 180-degree angle minus twice the rotation value previously obtained. This ensures that the tail is correctly flipped relative to the axis origin.

In C++, the implementation is done inside the AmmoItem class. For flexibility purposes, it triggers inside the Update loop—that way, it is possible to modify the trajectory of the shell and not worry about turning it again manually after launch:

if ((Velocity.x > 0 && Velocity.y > 0) || (Velocity.x > 0 && Velocity.y <0))
{
    Rotation = CalculateRadians(Velocity);
    Rotation += CalculateRadiansFromAngle(180) - 2 * Rotation;
}
else
{
    Rotation = CalculateRadians(Velocity);
}

CalculateRadians is the method that transforms the velocity components in a rotation value. It is located in the BasicMath.h helper:

inline float CalculateRadians(float2 velocity)
{
    float hypothenuse = sqrt(velocity.x * velocity.x + velocity.y * velocity.y);
    float sine = velocity.y / hypothenuse;
    float angle = asin(sine);
    return angle;
};

There is a potential problem with the implementation above. As the user taps on different parts of the screen, the X and Y components are different, each resulting in a different hypotenuse. As the target is set, the shell flies faster the further away from the bear the user taps. To avoid this, the triangle legs should be normalized to a near-constant value:

Take a look at the ShootAtTarget function in the Bear class:

void Bear::ShootAtTarget(float2 lastTargetTrace)
{
    OnPulledTrigger(Position.x, Position.y, 
    GetVelocityLegs(lastTargetTrace).x, GetVelocityLegs(lastTargetTrace).y, 
    CurrentDamage, IsFriendly, false, HostContainer->CurrentSpriteBatch);
}

Notice that the triangle legs are not passed as a raw value, but are proxied through GetVelocityLegs, which forces the resulting vector to be produced from a constant triangle with the velocity constant at 10 pixels per iteration:

float2 LivingEntity::GetVelocityLegs(float2 lastTargetTrace)
{
    float bottomLeg = 0.0f;
    float sideLeg = (Position.y - lastTargetTrace.y) / 100.0f;
    bottomLeg = (Position.x - lastTargetTrace.x) / 100.0f;
    float requiredVelocity = 10.0f;
    float hypothenuse = sqrt(bottomLeg * bottomLeg + sideLeg * sideLeg);
    float proportionalX = 0.0f;
    float proportionalY = 0.0f;
    proportionalX = (requiredVelocity * bottomLeg) / hypothenuse;
    proportionalY = (sideLeg < 0.0f ? -1 : 1) * sqrt(requiredVelocity * requiredVelocity - proportionalX * proportionalX);
    return float2(proportionalX,proportionalY);
}

For ammo that does not have a visual rotation dependency, such as the plasma ball, simply increment the rotation to make the thrown item continuously spin:

if (Type == PowerupType::PLASMA_BALL)
{
    if ((Velocity.x > 0 && Velocity.y > 0) || (Velocity.x > 0 && Velocity.y <0))
    {
        Rotation = CalculateRadians(Velocity);
        Rotation += CalculateRadiansFromAngle(180) - 2 * Rotation;
    }
    else
    {
        Rotation = CalculateRadians(Velocity);
    }
}
else
{
    Rotation += 0.2f;
}

Conclusions

FallFury mostly relies on sprite-based animations in which there are several textures cycled through in order to create the desired dynamic effect. These are not really hard to build, given that there is a possibility to integrate them in a timed loop. Be aware, however, that with slower machines the timing might be off and the time delta value between loops might be higher. In that case the sprite cycling will not be as smooth as it should be, which is why it’s important to perform animation testing on a variety of hardware with different OS loads.

In the previous article, you learned how to build level XML files in order to create playable levels. This article discusses how the internal parser works and how XML nodes become items on the game screen.  Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-6-Rendering-Level-Elements.  For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

Project Interop – The C# Parser

Thanks to classes such as XmlSerializer and XDocument, parsing XML in .NET Framework is not a complicated task. Similarly, in the WinRT world there are alternatives such as Windows::Data::Xml::Dom. That being said, using the parser skeleton in C# this project leverages the existing codebase, adapting it to the specific level reading needs.

Start by creating a new WinRT project that is the part of the existing solution. Make sure that you create a Windows Runtime Component:

There are major differences between the .NET and WinRT stacks, especially when it comes to building code that can be invoked from any potential WinRT project type, as is the case here. Make sure that the output of the newly created project is a Windows Runtime Component (WinMD file):

The XMLReader project reads much more than the level data—it also manages internal XML metadata such as user scores. This article, however, focuses only on level-related aspects of the engine.

Reader is the one core class used here. It contains the ReadXml method, which receives the file name and the type of the XML file to be read. Regardless of the name, the XML file type determines the parsing rules, the root lookup location, and the produced output. The tier (level set) and individual level data are stored internally in the application folder, and the score metadata is outside the sandbox in the application data folder:

async Task<Object> ReadXml(string fileName, XmlType type)
{
        StorageFile file = null;
        StorageFolder folder = null;
    
        if (type == XmlType.LEVEL || type == XmlType.TIERS)
        {
                folder = Windows.ApplicationModel.Package.Current.InstalledLocation;
                folder = await folder.GetFolderAsync("Levels");
        }
        else if (type == XmlType.HIGHSCORE)
        {
                folder = ApplicationData.Current.LocalFolder;
                folder = await folder.GetFolderAsync("Meta");
        }
    
        file = await folder.GetFileAsync(fileName);
    
        Object returnValue = null;
        string data;
    
        using (IRandomAccessStream readStream = await file.OpenAsync(FileAccessMode.Read))
        {
                using (Stream inStream = Task.Run(() => readStream.AsStreamForRead()).Result)
                {
                        using (StreamReader reader = new StreamReader(inStream))
                        {
                                data = reader.ReadToEnd();
                        }
                }
        }
    
        if (type == XmlType.LEVEL)
        {
                returnValue = ExtractLevelData(data);
        }
        else if (type == XmlType.HIGHSCORE)
        {
                returnValue = ExtractScoreData(data);
        }
        else if (type == XmlType.TIERS)
        {
                returnValue = ExtractTierData(data);
        }
    
        return returnValue;
}

Before the level data is extracted, FallFury needs to know about the existing tiers pointing to those levels. That’s where ExtractTierData, an internal method that accepts a raw XML string and return a TierSet instance, comes in:

public sealed class TierSet
{
    public Tier[] Tiers { get; set; }
}

A Tier class has the following structure:

public sealed class Tier
{
    public string Name { get; set; }
    public string[] LevelNames { get; set; }
    public string[] LevelFiles { get; set; }
}

I am using the most fundamental types – arrays instead of generic collections, to maximize the portability of the code. The ExtractTierData method handles the XML-to-Object transformation:

TierSet ExtractTierData(string tierString)
{
        TierSet set = new TierSet();
    
        XDocument document = XDocument.Parse(tierString);
    
        set.Tiers = new Tier[document.Root.Elements().Count()];
    
        int tierCounter = 0;
    
        foreach (XElement element in document.Root.Elements())
        {
                Tier tier = new Tier();
                tier.Name = element.Attribute("name").Value;
        
                tier.LevelFiles = new string[element.Elements("level").Count()];
                tier.LevelNames = new string[element.Elements("level").Count()];
        
                
                int count = 0;
                foreach (XElement lElement in element.Elements("level"))
                {
                        tier.LevelFiles[count] = lElement.Attribute("file").Value;
                        tier.LevelNames[count] = lElement.Attribute("name").Value;
                        count++;
                }
        
                set.Tiers[tierCounter] = tier;
                tierCounter++;
        }
    
        return set;
}

I am running a coupled array block here, utilizing one array for level names and another for file locations. This is a very basic implementation of a key-value pair that depends on code-based coupling. ExtractTierData, however, is not exposed publicly and the C++ layer of FallFury will not access it because TierSet is not exposed through a compatible async method. Looking back at ReadXml, might seem like the answer to this problem. But a Task<Object> is not an interop-compatible type, which means I use ReadXmlAsync, which proxies the call through an IAsyncOperation:

public IAsyncOperation<Object> ReadXmlAsync(string filename, XmlType type)
{
    return (IAsyncOperation<Object>)AsyncInfo.Run((CancellationToken token) => ReadXml(filename, type));
}

AsyncInfo.Run starts a WinRT async operation and handles the returned result, regardless of the selected file path or type.

Level-based data is extracted in a similar manner to the tier data. The difference lies in the used internal models, as well as the node reading order:

Level ExtractLevelData(string levelString)
{
        XDocument document = XDocument.Parse(levelString);
    
        Level level = new Level();
        level.LevelMeta = new Meta();
     
        level.LevelMeta.Score = Convert.ToInt32(document.Root.Element("meta").Attribute("score").Value);
        level.LevelMeta.ButtonPrice = Convert.ToInt32(document.Root.Element("meta").Attribute("buttonPrice").Value)
        level.LevelMeta.LevelType = (LevelType)Convert.ToInt32(document.Root.Attribute("type").Value);
    
        int count = document.Root.Element("obstacles").Elements().Count();
        level.Obstacles = new Obstacle[count];
    
        count = 0;
        foreach (XElement element in document.Root.Element("obstacles").Elements())
        {
                Obstacle obstacle = new Obstacle();
                obstacle.HealthDamage = Convert.ToSingle(element.Attribute("healthDamage").Value);
                obstacle.InflictsDamage = Convert.ToBoolean(element.Attribute("inflictsDamage").Value);
                obstacle.Rotation = Convert.ToSingle(element.Attribute("rotation").Value);
                obstacle.Scale = Convert.ToSingle(element.Attribute("scale").Value);
                obstacle.X = Convert.ToSingle(element.Attribute("x").Value);
                obstacle.Y = Convert.ToSingle(element.Attribute("y").Value);
                obstacle.Type = (ObstacleType)Convert.ToInt32(element.Attribute("type").Value);
                level.Obstacles[count] = obstacle;
                count++;
        }
    
        count = document.Root.Element("monsters").Elements().Count();
        level.Monsters = new Monster[count];
    
        count = 0;
        foreach (XElement element in document.Root.Element("monsters").Elements())
        {
                Monster monster = new Monster();
                monster.CriticalDamage = Convert.ToSingle(element.Attribute("criticalDamage").Value);
                monster.Damage = Convert.ToSingle(element.Attribute("damage").Value);
                monster.DefaultAmmo = Convert.ToInt32(element.Attribute("defaultAmmo").Value);
                monster.MaxHealth = Convert.ToSingle(element.Attribute("maxHealth").Value);
                monster.X = Convert.ToSingle(element.Attribute("x").Value);
                monster.Y = Convert.ToSingle(element.Attribute("y").Value);
                monster.VelocityX = Convert.ToSingle(element.Attribute("velocityX").Value);
                monster.Lifetime = Convert.ToSingle(element.Attribute("lifetime").Value);
                monster.Type = (MonsterType)Convert.ToInt32(element.Attribute("type").Value);
                monster.Bonus = Convert.ToInt32(element.Attribute("bonus").Value);
                monster.Scale = Convert.ToSingle(element.Attribute("scale").Value);
        
                level.Monsters[count] = monster;
                count++;
        }
    
        count = document.Root.Element("buttons").Elements().Count();
        level.Buttons = new Button[count];
    
        count = 0;
        foreach (XElement element in document.Root.Element("buttons").Elements())
        {
                Button button = new Button();
        
                button.X = Convert.ToSingle(element.Attribute("x").Value);
                button.Y = Convert.ToSingle(element.Attribute("y").Value);
        
                level.Buttons[count] = button;
                count++;
        }
    
        count = document.Root.Element("powerups").Elements().Count();
        level.Powerups = new Powerup[count];
    
        count = 0;
        foreach (XElement element in document.Root.Element("powerups").Elements())
        {
                Powerup powerup = new Powerup();
        
                powerup.X = Convert.ToSingle(element.Attribute("x").Value);
                powerup.Y = Convert.ToSingle(element.Attribute("y").Value);
                powerup.Category = (PowerupCategory)Convert.ToInt32(element.Attribute("category").Value);
                powerup.Type = (PowerupType)Convert.ToInt32(element.Attribute("type").Value);
                powerup.Lifespan = Convert.ToSingle(element.Attribute("lifespan").Value);
                powerup.Effect = Convert.ToSingle(element.Attribute("effect").Value);
        
                level.Powerups[count] = powerup;
                count++;
        }
    
        Bear bear = new Bear();
        XElement bearElement = document.Root.Element("bear");
        bear.CriticalDamage = Convert.ToSingle(bearElement.Attribute("criticalDamage").Value);
        bear.Damage = Convert.ToSingle(bearElement.Attribute("damage").Value);
        bear.DefaultAmmo = Convert.ToInt32(bearElement.Attribute("defaultAmmo").Value);
        bear.MaxHealth = Convert.ToSingle(bearElement.Attribute("maxHealth").Value);
        bear.StartPosition = Convert.ToSingle(bearElement.Attribute("startPosition").Value);
        bear.Velocity = Convert.ToSingle(bearElement.Attribute("velocity").Value);
    
        level.GameBear = bear;
        
        return level;
}

As a result, the Level instance has an internal counterpart in the C++ project:

public sealed class Level
{
    public Bear GameBear { get; set; }
    public Meta LevelMeta { get; set; }
    public Obstacle[] Obstacles { get; set; }
    public Monster[] Monsters { get; set; }
    public Button[] Buttons { get; set; }
    public Powerup[] Powerups { get; set; }
}

From C# to C++ - Leveraging the WinRT Component

At this point, the C#-based WinRT component is complete and can be integrated into the C++ project. This can be done in two ways: either by adding a reference to the project itself or by adding a reference to the generated WinMD file. Both methods will ultimately produce the same output, but it’s easier to debug and modify coupled projects on the go, so I went with the first option.

To add a reference to an internal Windows Runtime Component project, right click on the C++ project in Solution Explorer and select Properties. You will see a dialog like this:

Select the Common Properties node in the tree on the left and open the Framework and References page:

In the sample screen capture above, the FallFury.XMLReader project is already added as a reference. For a new project, simply click on Add New Reference and add any compatible project or third-party extension. As soon as the reference is added, the publicly exposed methods can be accessed.

In DirectXPage.cpp I have a method called LoadLevelData that allows me to load the list of the registered levels from core.xml as well as build the visual tree for the menu items, whichallows players to select a level:

void DirectXPage::LoadLevelData()
{
    Coding4Fun::FallFury::XMLReader::Reader^ reader = ref new Coding4Fun::FallFury::XMLReader::Reader();
    Windows::Foundation::IAsyncOperation<Platform::Object^>^ result = reader->ReadXmlAsync("core.xml", Coding4Fun::FallFury::XMLReader::Models::XmlType::TIERS);
    result->Completed =
        ref new AsyncOperationCompletedHandler<Platform::Object^>(this, &DirectXPage::OnLevelLoadCompleted);
}

Following the normal asynchronous pattern, as well as the structure of the method exposed in the Reader class, I am calling ReadXmlAsync and getting the result in OnLevelLoadCompleted when the operation reaches its final stage. It is worth mentioning that the associated AsyncOperationCompletedHandler is invoked even when the reading fails; therefore, the invocation of that callback does not on its own mean that the necessary data is obtained:

Here is what happens when OnLevelLoadCompleted is called:

void DirectXPage::OnLevelLoadCompleted(IAsyncOperation<Platform::Object^> ^op, AsyncStatus s)
{
    if (s == AsyncStatus::Completed)
    {
        auto set = (Coding4Fun::FallFury::XMLReader::Models::TierSet^)op->GetResults();
        
        auto tiers = set->Tiers;
        
        int levelCounter = 0;
        
        for (auto tier = tiers->begin(); tier != tiers->end(); tier++)
        {
            StackPanel^ panel = ref new StackPanel();
            
            TextBlock^ levelTierTitle = ref new TextBlock();
            levelTierTitle->Text = (*tier)->Name;
            levelTierTitle->Style = (Windows::UI::Xaml::Style^)Application::Current->Resources->Lookup("LevelSelectTierItemText");
            levelTierTitle->RenderTransform = ref new TranslateTransform();
            panel->Children->Append(levelTierTitle);
            
            levelNames = (*tier)->LevelNames;
            auto levelFiles = (*tier)->LevelFiles;
            
            int max = levelNames->Length;
            
            for(int i = 0; i < max; i++)
            {
                MenuItem^ item = ref new MenuItem();
                item->Tag = levelCounter;
                item->Label = levelNames[i];
                item->HorizontalAlignment = Windows::UI::Xaml::HorizontalAlignment::Left;
                item->OnButtonSelected += ref new MenuItem::ButtonSelected(this, &DirectXPage::OnLevelButtonSelected);
                item->RenderTransform = ref new TranslateTransform();
                
                m_renderer->Levels.Insert(levelCounter,levelFiles[i]);
                
                panel->Children->Append(item);
                levelCounter++;
            }
            
            stkLevelContainer->Items->Append(panel);
        }
    }
}

Here, AsyncStatus can also be Cancelled or Error, so checking for Complete ensures that the expected result is processed further inside the context of the callback.

As the returned TierSet exposes the Tiers array, I am simply iterating through each of the existing items to create independent tier blocks, grouped in StackPanel elements, coupled with level-specific MenuItem instances that present the user with a choice of playing a given level. The level ID is carried in the Tag property and it will be used to identify the selected button when DirectXPage::OnLevelButtonSelected is triggered:

Reading Level-specific Data

Level data is read in the GamePlayScreen once the selected menu button passes the name and level identifiers. The LoadLevelXml method is called as the screen begins to load, preparing all assets for the start of the game session:

void GamePlayScreen::LoadLevelXML()
{
    Coding4Fun::FallFury::XMLReader::Reader^ reader = ref new Coding4Fun::FallFury::XMLReader::Reader();
    
    Platform::String^ LevelName = Manager->Levels.Lookup(StaticDataHelper::CurrentLevelID);
    
    Windows::Foundation::IAsyncOperation<Platform::Object^>^ result = 
            reader->ReadXmlAsync(LevelName, Coding4Fun::FallFury::XMLReader::Models::XmlType::LEVEL);
    
        result->Completed =
            ref new AsyncOperationCompletedHandler<Platform::Object^>(this, &GamePlayScreen::OnLevelLoadCompleted);
}

The selected ID passes from the menu screen to the game screen through CurrentLevelID, an intermediary value preserved in a StaticDataHelper class. The level file name is looked up based on the ID and then passed to ReadXmlAsync with the XmlType set to LEVEL. When the load completes, OnLevelLoadCompleted is invoked, and an additional helper class— LevelDataLoader—sets up the game components based on the received data:

void GamePlayScreen::OnLevelLoadCompleted(IAsyncOperation<Platform::Object^> ^op, AsyncStatus s)
{
        if (s == AsyncStatus::Completed)
        {
                InitializeSpriteBatch();    
        m_loader = ref new BasicLoader(Manager->m_d3dDevice.Get(), Manager->m_wicFactory.Get());
        CreateBear();
        
                LevelDataLoader^ loader = ref new LevelDataLoader((Coding4Fun::FallFury::XMLReader::Models::Level^)op->GetResults(), this);
        loader->SetupBear(GameBear);
        loader->SetupObstacles(m_obstacles);
        loader->SetupMonsters(m_monsters);
        loader->SetupButtons(m_buttons, m_buttonPrice);
        loader->SetupPowerups(m_powerups);
        m_currentLevelType = (LevelType) loader->CurrentLevel->LevelMeta->LevelType;
        
        StaticDataHelper::CurrentLevel = loader->CurrentLevel;
        StaticDataHelper::ButtonsTotal = loader->CurrentLevel->Buttons->Length;
        
        LoadTextures();
        
                CreateMonster();
                CreatePowerups();
        
        IsLevelLoaded = true;
        
        GameBear->TurnRight();
        m_particleSystem.CreatePreCachedParticleSets();
        LoadSounds();
        }
}

Each object is separated in its own method, such as SetupBear or SetupObstacles. SetupBear, for example, transforms the exposed managed Bear model to a native C++ Characters::Bear one:

void LevelDataLoader::SetupBear(Bear ^gameBear)
{
        gameBear->Position = float2(GetXPosition(CurrentLevel->GameBear->StartPosition), 0);    
    gameBear->MaxHealth = CurrentLevel->GameBear->MaxHealth;
    gameBear->CurrentHealth = CurrentLevel->GameBear->MaxHealth;
    gameBear->CurrentDamage = CurrentLevel->GameBear->Damage;
    gameBear->Velocity.y = CurrentLevel->GameBear->Velocity;
    gameBear->MaxCriticalDamage = CurrentLevel->GameBear->CriticalDamage;
    gameBear->Rotation = 0.0f;
}

Conclusion

Mixing C# and C++ components is not a complicated process. Nonetheless, it comes with specific restrictions and considerations, such as the format of the publicly exposed asynchronous calls. The above C#-based level loading engine highlights the fact that WinRT interoperability allows you to leverage the languages you know best in order to efficiently accomplish project tasks.

One of the goals set when developing FallFury was making the game extensible in regards to playable levels. Furthermore, I thought that it would make sense from the development and testing perspective to have levels as a completely separate entity that can be modified as necessary. For example, when a new power-up was introduced, I wanted to add an extra line in a level file and test it out. This was ultimately achieved by creating an XML-based level engine and in this article I will describe the level structure and design process.

Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-5-Creating-Levels.  For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

The First Steps

When I started working on the level engine concept, I began designing the potential XML file structure for the level and ended up with the following requirements:

• Level Type Identifier – As different levels have different backgrounds and sound themes, there should be a way to mark a level type. There are currently four level types: dream, nightmare, space, and magic bean.
• The Starting Character Descriptor – When the game starts, the teddy bear has some initial, basic properties, such as maximum health level, horizontal position, and velocity.
• A Collection of Obstacles – For each level, obstacles are positioned differently, and it’s important to specify that. For some obstacles it might be desirable to disable the damage infliction component, while for others it might be good to maximize the damage caused by colliding with them. Also, there are multiple textures associated with different obstacle types, so I wanted to specify the obstacles to render regardless of the selected level type.
• A Collection of Monsters – Obstacles are not the only component that can damage the bear during gameplay. There are also monsters that can pop up and shoot at the main character. Similar to the bear, monsters represent a living entity and have some specific properties, such as the initial health, damage, starting position, velocity, and type.
• Buttons – These are the bonus point boosters in FallFury. The player collects as many of those as possible, and each of them should be individually positioned to form either a trail or a shape.
• Power-Ups – With the basic set of abilities, the bear is able to get some bonuses such as a cape that will speed-up his descent or a bubble that will protect him from incoming shells.

The first build of the level engine integrated into FallFuryused percentage-based relative values to position elements on the screen. Although this seemed like a good idea at the time, it became problematic because

• It required the level to be a fixed size, which restricted element addition and level extension.
• It caused problems with obstacles that needed to be scaled and therefore had a non-standard size.
• Small modifications were harder to make because minimal adjustments would throw off the relative position.

So, I switched to a pixel-based conditioning in which each position is relative to zero. With this in place, levels can be infinitely long (within the context of the machine’s rendering and memory capabilities)and extra elements can be more seamlessly added.

Additionally, levels need to be packaged together in individual sets, normally grouped by themes, without restriction. This is achieved with the help of an extra XML file,called core.xml, which keeps track of level tiers, and acts as a container that allows the developer to name and easily enable or disable specific levels .

The structure for the core.xml file looks like this:

<tiers>
  <tier name="GO, GO, TEDDY">
    <level name="mind travels" file="GoGoTeddy\mind.xml"></level>
    <level name="falling in" file="Nightmare\full_pilot.xml"></level>
    <level name="frontlines" file="Nightmare\the_beginning.xml"></level>
    <level name="the chase" file="Nightmare\chasing_monsters.xml"></level>
  </tier>
  <tier name="SECRET GARDEN">
    <level name="bean stalking" file="Garden\bean_stalking.xml"/>
    <level name="thorn apart" file="Garden\thorn_apart.xml"/>
  </tier>
  <!--<tier name="Obstacle ***TEST***">
    <level name="Nightmare 0" file="test\Obstacle\nightmare\0.xml" />
    <level name="Nightmare 1" file="test\Obstacle\nightmare\1.xml" />
    <level name="Bean 0" file="test\Obstacle\bean\0.xml" />
    <level name="Bean 1" file="test\Obstacle\bean\1.xml" />
    <level name="Dream 0" file="test\Obstacle\dream\0.xml" />
    <level name="Dream 1" file="test\Obstacle\dream\1.xml" />
  </tier>-->
  <!--<tier name="Death ***TEST***">
    <level name="Monster 0" file="test\death\0.xml" />
  </tier>-->
  <!--<tier name="Monster ***TEST***">
    <level name="0" file="test\Monsters\0.xml" />
    <level name="1" file="test\Monsters\1.xml" />
    <level name="2" file="test\Monsters\2.xml" />
    <level name="3" file="test\Monsters\3.xml" />
    <level name="4" file="test\Monsters\4.xml" />
    <level name="5" file="test\Monsters\5.xml" />
    <level name="6" file="test\Monsters\6.xml" />
    <level name="7" file="test\Monsters\7.xml" />
    <level name="8" file="test\Monsters\8.xml" />
    <level name="9" file="test\Monsters\9.xml" />
    <level name="10" file="test\Monsters\10.xml" />
  </tier>-->
  <!--<tier name="MEDALS *****TEST******">
    <level name="gold" file="test\Medals\gold.xml"></level>
    <level name="silver" file="test\Medals\silver.xml"></level>
    <level name="bronze" file="test\Medals\bronze.xml"></level>
  </tier>
  <tier name="Buttons ***TEST***">
    <level name="1" file="test\Buttons\single.xml" />
    <level name="Lots" file="test\Buttons\multiple.xml" />
  </tier>
  <tier name="Obstacles ***TEST***">
    <level name="Cape" file="test\PowerUps\0.xml" />
  </tier>-->
</tiers>

Tiers that are commented out are ignored and the included levels aren’t on the game list. Also, the paths indicated for each file attribute—for each individual tier—are relative to the game folder itself. There is no limit on the number of subfolders that can be included in the path. The above structure will render this level set:

The Level XML

Let’s now take a look at the layout of the level descriptor XML file:

<?xml version="1.0" encoding="utf-8" ?>
<level type="0">
  <meta score="0" buttonPrice="10"></meta>
  <bear maxHealth="100" startPosition="300" velocity="8.0" damage="11" criticalDamage="20" defaultAmmo="100" />
  <obstacles>
    <obstacle type="1" x="119" y="2300" inflictsDamage="true" healthDamage="5" rotation="3.14" scale="1" />
    <obstacle type="3" x="534.5" y="3000" inflictsDamage="true" healthDamage="5" rotation="0" scale="1" />
    <obstacle type="3" x="534.5" y="3300" inflictsDamage="true" healthDamage="5" rotation="0" scale="1" />
    <obstacle type="1" x="119" y="4200" inflictsDamage="true" healthDamage="5" rotation="3.14" scale="1" />
    <obstacle type="2" x="546" y="5000" inflictsDamage="true" healthDamage="5" rotation="0" scale="1" />
    <obstacle type="1" x="119" y="5800" inflictsDamage="true" healthDamage="5" rotation="3.14" scale="1" />
    <obstacle type="2" x="546" y="6600" inflictsDamage="true" healthDamage="5" rotation="0" scale="1" />
  </obstacles>
  <monsters>
    <monster lifetime="3000" scale=".2" velocityX="2" velocityY="2" type="0" x="460" y="19900" maxHealth="80" bonus="100" lives="0" damage="10" criticalDamage="8" defaultAmmo="50" />
    <monster lifetime="3000" scale=".2" velocityX="2" velocityY="2" type="1" x="460" y="25000" maxHealth="80" bonus="100" lives="0" damage="10" criticalDamage="8" defaultAmmo="50" />
    <monster lifetime="3000" scale=".2" velocityX="2" velocityY="2" type="2" x="460" y="34500" maxHealth="80" bonus="100" lives="0" damage="10" criticalDamage="8" defaultAmmo="50" />
    <monster lifetime="6000" scale=".4" velocityX="2" velocityY="2" type="3" x="460" y="41400" maxHealth="180" bonus="100" lives="0" damage="17" criticalDamage="8" defaultAmmo="50" />
  </monsters>
  <buttons>
    <button x="300" y="800" />
    <button x="360" y="800" />
    <button x="300" y="860" />
    <button x="360" y="860" />
    <button x="300" y="920" />
    <button x="360" y="920" />
    <button x="300" y="980" />
    <button x="360" y="980" />
  </buttons>
  <powerups>
    <powerup category="1" type="4" x="140" y="9200" effect="3" lifespan="4"></powerup>
    <powerup category="1" type="3" x="480" y="19800" effect="10" lifespan="6"></powerup>
    <powerup category="1" type="0" x="480" y="27500" effect="10" lifespan="6"></powerup>
    <powerup category="1" type="1" x="100" y="34000" effect="10" lifespan="6"></powerup>
    <powerup category="1" type="0" x="100" y="41200" effect="10" lifespan="6"></powerup>
  </powerups>
</level>

The opening level tag carries a type attribute. This is level theme flag. It can be set to one of the three four values:

• 0 – The Nightmare Theme
• 1 – The Magic Bean Theme
• 2 – The Dream Theme
• 3 – The Space Theme

You can see the design differences in the images below:

Remember, that obstacles and the level theme itself do not influence much other than the background and soundboard.

Once the type is specified, the meta tag brings up the score and buttonPrice attributes. If you for some reason want to create a level including an initial score, you can specify it here. And because buttons are fixed bonus assets that are all created equal, each of them carries a given bonus point weight. The score based on the collected buttons is calculated at the end of the game and relies on the value specified in the meta tag.

Obstacles

Next comes the obstacle collection, which is represented by the obstacles tag. This tag is required even if there are no obstacles on a given level. Simply use <obstacles /> as necessary. Each child node represents an instance of an obstacle that can be choosen from the following enum:

enum class ObstacleType
{
    OT_CLOUD = 0,
    OT_SPIKE_NIGHTMARE_LARGE = 1,
    OT_SPIKE_NIGHTMARE_MEDIUM = 2,
    OT_SPIKE_NIGHTMARE_SMALL = 3,
    OT_BEAN_A = 4,
    OT_BEAN_B = 5,
    OT_BEAN_C = 6,
    OT_BEAN_D = 7,
    OT_BEAN_E = 8,
    OT_SPACE_ROCKET = 9,
    OT_SPACE_COMET_A = 10,
    OT_SPACE_COMET_B = 11,
    OT_SPACE_SATELLITE = 12,
    OT_SPACE_UFO = 13,
    OT_SPACE_BALL = 14
};

Here is the complete table showing the appearance of each of them:

No matter how the obstacles are positioned, they will either be located in the visible area or displaced outside the viewport and not displayed. The ultimate position is taken from the obstacle size and is relative to the center of the texture. For example, if an obstacle texture is 400- pixels wide, the X-relative position should be set to 200. If the position differs from the starting one, however, the obstacle texture is cut to include the area that fits in the 768 pixel wide playable zone visible. There are no restrictions regarding the Y position.

To help in level creation, some obstacles have pre-defined left and right margins. For example:

• OT_BEAN_A: 269.5 (Left), 498.5 (right, with 3.14 rotation)
• OT_BEAN_B: 128.5 (left), 638.5 (right, with 3.14 rotation)
• OT_BEAN_C: 172 (left), 596 (right, with 3.14 rotation)
• OT_BEAN_D: 188.5 (left), 579.5 (right, with 3.14 rotation)
• OT_BEAN_E: 206.5 (left), 561.5 (right, with 3.14 rotation)

The inflictsDamage attribute determines whether the obstacle harms the main character. If it is set to false, the character will still make the sound of colliding with it but will not lose any health points. The primary use for this attribute is level testing.If it is set to true, the character will loose the amount of health points indicated by the healthDamage attribute.

The rotation and scale attributes can be used to flip and resize the texture as needed. Rotation is measured in radians, and the scale is a normalized value in which 1.0 represents 100% of the scale.

Monsters

As with obstacles, the monsters node should never be omitted from the file and should at least contain a placeholder: <monsters />. Unlike obstacles, however, monsters are dynamic and do not have fixed positions. Moreover, monsters have limited active time during gameplay. The first attribute,lifetime, determines the length of the fall during which the monster will be visible in the viewport. With the y attribute as the Y-based position at which the monster appears, at the y+lifetime position the monster simply flies away if not killed.

The scale attribute carries the same purpose as the one for the obstacle—normalized texture size relative to the size of the original image file. As such, the level designer does not have to worry about linking the width and height of the monster when resizing and can instead use a percentage-like value to scale the monster up or down, simultaneously modifying both the width and the height with zero stretching.

Nextup are velocityX and velocityY. These two attributes are used to set the motion velocity when the monster is already visible. Instead of being a static shooting entity, the enemy moves on a randomized zig-zag path at the bottom of the screen. The horizontal and vertical displacement—in pixels, per update cycle—is individually set through the values carried by the above-mentioned attributes. If necessary, this functionality can be disabled in the code-behind by assigning a fixed value for the vertical and horizontal movement for all monsters that are being loaded on a given level.

The monster type is an integer value that is translated in a value from the following enum (located in MonsterType.h):

enum class MonsterType
{
    MT_NIGHTMARE_A = 0,
    MT_NIGHTMARE_B = 1,
    MT_NIGHTMARE_C = 2,
    MT_MAGICBEAN_A = 3,
    MT_MAGICBEAN_B = 4,
    MT_MAGICBEAN_C = 5,
    MT_CANDYLAND_A = 6,
    MT_CANDYLAND_B = 7,
    MT_CANDYLAND_C = 8,
    MT_CANDYLAND_D = 9,
    MT_CANDYLAND_E = 10
};

Here is a table that shows the texture associated with each identifier:

Each monster has three separate textures associated with it. The three textures are cycled inside the update loop for each monster entity when the monster becomes visible, creating the movement effect:

Each monster currently shoots only one type of ammo: a red plasma ball. Be aware, however, that there is a preprogrammed condition in which the last monster in the XML file collection is automatically considered the final boss. This means the scale, maxHealth and lifetime properties must be manually adjusted to reflect the effect. Without doing so, the last monster will, regardless of the XML setting, switch in-game to a triple fireball shot that inflicts three times the damage indicated by the damage attribute:

Each monster can have limited ammo as set by the defaultAmmo attribute. In the case that the ammo is exhausted before the monster expires, the monster will continue its motion at the bottom of the screen without inflicting direct damage to the main character.

Buttons

These are the least complex entities and only carry an X and a Y position. Given those coordinates, relative to the left margin of the visible area, a button texture is rendered:

Once picked-up, the button counter is increased by one and the meta-score incremented by the value set in the meta tag at the beginning of the level XML as long as the feature is activated in the code-behind.

Power-ups

To make the game more fun, there are bonus elements that can be picked up by the bear in order to enhance its performance or protection. These elements are declared in the <powerups/> collection. First and foremost, it is important to declare whether the power-up is positive or negative. In the current version of FallFury, only positive power-ups are included. Nonetheless, the harness for negative ones is already integrated in the parser. Therefore, the category attribute should be set to 1 if the power-up has a positive effect and 0 for a negative effect. This value will only have an effect over the sound played when the bonus is collected.

The power-up type can be one of the following (enum located in PoweupType.h):

enum class PowerupType
{
    HEALTH = 0,
    HELMET = 1,
    PARACHUTE = 2,
    BUBBLE = 3,
    CAPE = 4,
    AXE = 5,
    BOOMERANG = 6,
    HAMMER = 7,
    KNIFE = 8,
    PLASMA_BALL = 9,
    CIRCLE = 10
};

The table below shows the power-up texture appearance. Behaviors are already defined in the game and influenced by only the effect and lifespan (seconds) attributes:

CIRCLE is a helper power-up that has no effect on the bear and is instead used as an additional texture overlay along with any other power-up in order to create a pulsating circle effect.

Conclusion

FallFury ships with a dozen of sample levels that showcase all of the elements described in the article. At the moment, a level editor is in the works, but it isn’t too complicated to build XML files manually. To do so, you need to consider the pixel-based locations and ensure that they’re all in the visible area—the game engine will automatically handle all other displacements and adjustments.

The Concept of a Swap Chain

Before I go into detail about the DirectX and XAML interop in FallFury, I want to cover one important aspect of DirectX development that you need to familiarize yourself with: the swap chain.

When your graphics adapter draws on the visual surface, you, as the user, see only minor potential redraws. Internally, however, the device switches buffers that reflect the displayed content, with each buffer representing a frame that has to be drawn. You can deduce from this that any swap chain has at least two buffers that it can switch between.

For example, if I want to display my character as being displaced by a specific amount of pixels, the buffer will at the outset present to me the character in its initial position, while the second buffer will be constructed in the background with the proper position adjustments. The first frame, made from the content from the first buffer, will be discarded, and then the second frame will be displayed, and so on. This process occurs at a very high speed that depends on the processing capabilities of the graphics adapter, so the user does not notice the swapping itself.

The most common swap chain is composed of two buffers—the screenbuffer and the secondary framebuffer.

SwapChainBackgroundPanel

DirectX interoperability with XAML simplifies a lot of routine tasks that would otherwise be handled with manual rendering procedures, such as a menu system or a simple game HUD. That being said, the way the XAML workflow is organized in a Direct2D project is quite different compared to a standard XAML Windows Store or Windows Phone application.

The core difference is that there is no navigational system per-se and the fundamental entity in a Direct2D (XAML) project that manages the XAML content is a SwapChainBackgroundPanel control. This control allows the developer to overlay XAML elements on top of the DirectX renders. It replaces the normal page-based layout with one in which it is sole container for every XAML control that has to be used in the application. This necessitates that you will have to organize the secondary elements in such a way that the correct set is displayed for the current application state.

For example, if the user is in the main menu, menu options as well as the game logo should be shown. When the user switches to the game mode, the HUD should appear and the menu should become hidden. Although both the menu and the HUD are a part of the same SwapChainBackgroundPanel, I will have to manually manage state and visibility changes.

Using the SwapChainBackgroundPanel also means that you will have to enforce specific graphic configuration rules in your application. One of them applies to setting up the swap chain. When you set up the scaling, it must be set to DXGI_SCALING_STRETCH:

DXGI_SWAP_CHAIN_DESC1 swapChainDesc = {0};
swapChainDesc.Width = static_cast<UINT>(m_renderTargetSize.Width); 
swapChainDesc.Height = static_cast<UINT>(m_renderTargetSize.Height);
swapChainDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
swapChainDesc.Stereo = false; 
swapChainDesc.SampleDesc.Count = 1;
swapChainDesc.SampleDesc.Quality = 0;
swapChainDesc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
swapChainDesc.BufferCount = 2;
swapChainDesc.Scaling = DXGI_SCALING_STRETCH;
swapChainDesc.SwapEffect = DXGI_SWAP_EFFECT_FLIP_SEQUENTIAL;
swapChainDesc.Flags = 0;

The swap chain itself should be configured for composition, mixing the native DirectX buffer with the overlaid XAML. This is done by calling CreateSwapChainForComposition:

ThrowIfFailed(
    dxgiFactory->CreateSwapChainForComposition(
        m_d3dDevice.Get(),
        &swapChainDesc,
        nullptr,
        &m_swapChain
        )
    );

ComPtr<ISwapChainBackgroundPanelNative> panelNative;
ThrowIfFailed(
    reinterpret_cast<IUnknown*>(m_panel)->QueryInterface(IID_PPV_ARGS(&panelNative))
    );

ThrowIfFailed(
    panelNative->SetSwapChain(m_swapChain.Get())
    );

There really isn’t much additional configuration work beyond this point. Remember, that the XAML content will always be overlaid on top of the DirectX content; therefore, plan the game components accordingly.

The XAML Menu System

The menu is at the core of the FallFury experience. When designing it, I was inspired by how Dance Central built the user interaction mechanism, and I tried to implement a similar approach in which the user would have to slide the button to the right instead of simply tapping on it:

Each menu item is built around a custom MenuItem control (MenuItem.xaml):

<UserControl
    x:Class="Coding4Fun.FallFury.MenuItem"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:Coding4Fun.FallFury"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    x:Name="menuItem"
    Margin="10, 0, 0, 10">

    <Grid 
        Height="65" 
        ManipulationMode="TranslateX"
        ManipulationCompleted="Grid_ManipulationCompleted"
        ManipulationDelta="Grid_ManipulationDelta" 
        x:Name="ControlContainer"
        PointerPressed="Grid_PointerPressed" 
        PointerReleased="Grid_PointerReleased">
        
        <Grid.Resources>
            <Storyboard x:Name="ArrowAnimator">
                <DoubleAnimation Storyboard.TargetName="ImageTranslateTransform"
                                 Storyboard.TargetProperty="X"
                                 From="0"
                                 To="20"
                                 Duration="0:0:0.4"
                                 RepeatBehavior="Forever"
                                 AutoReverse="True">
                </DoubleAnimation>
            </Storyboard>

        </Grid.Resources>
        
        <Grid.RowDefinitions>
            <RowDefinition Height="10" />
            <RowDefinition />
            <RowDefinition />
            <RowDefinition Height="10" />
        </Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="10" />
            <ColumnDefinition />
        </Grid.ColumnDefinitions>
        <StackPanel Orientation="Horizontal" Grid.RowSpan="4" Grid.ColumnSpan="4">
            <!-- width is sent in code behind, have to get this dynamic ...  -->
            <Grid Width="350" x:Name="coverRectangle">
                <Rectangle Fill="#303030" />
                <Rectangle 
                    Fill="Red" 
                    x:Name="coverActiveRectangle" />
            </Grid>
            <Image x:Name="MenuImage" Source="ms-appx:///MenuItems/single_arrow.png" Margin="10,0,0,0" Stretch="Uniform">
                <Image.RenderTransform>
                    <TranslateTransform x:Name="ImageTranslateTransform"></TranslateTransform>
                </Image.RenderTransform>
            </Image>
        </StackPanel>
        <!---->

        <TextBlock 
            Text="{Binding ElementName=menuItem, RelativeSource={RelativeSource Self}, Path=Label}"
            Grid.RowSpan="4"
            Grid.ColumnSpan="4"
            Style="{StaticResource MenuItemText}">

        </TextBlock>
        
        <MediaElement x:Name="coreMenuMedia" Source="ms-appx:///Assets/Sounds/MenuTap.wav" AutoPlay="False"></MediaElement>
        <MediaElement x:Name="slideMenuMedia" Source="ms-appx:///Assets/Sounds/MenuSlide.wav" AutoPlay="False"></MediaElement>
    </Grid>
</UserControl>

There is a core storyboard that is used to perform an image bounce. Initial tests showed that that most users tend to follow the standard “click-and-go” pattern and even though the implemented button was a slider, they tried to click it at least once, expecting the game to take them to the next screen. To avoid this, I introduced a visual arrow indicator that bounces left and right to indicate the direction of the necessary slide.

When the sliding occurs, the overlay grid width is being adjusted relative to the manipulation. At the same time, the button changes from passive state (dark background):

to active state (red background):

Internally, the double-arrow image replaces the single-arrow one and is translated across the X-axis to create a bouncing effect:

void MenuItem::Grid_PointerPressed(Platform::Object^ sender, Windows::UI::Xaml::Input::PointerRoutedEventArgs^ e)
{
    BitmapImage^ image = ref new BitmapImage(ref new Uri("ms-appx:///MenuItems/double_arrow.png"));
    MenuImage->Source = image;
    
    coverActiveRectangle->Visibility = Windows::UI::Xaml::Visibility::Visible;
    ((Storyboard^)ControlContainer->Resources->Lookup("ArrowAnimator"))->Begin();
    
    coreMenuMedia->Play();
}

As the user drags the finger on the button, the red overlay size should be adjusted accordingly. The size adjustment can be easily tracked in the ManipulationDelta event handler for the focused grid:

void MenuItem::Grid_ManipulationDelta(Platform::Object^ sender, Windows::UI::Xaml::Input::ManipulationDeltaRoutedEventArgs^ e)
{
    double diff = e->Cumulative.Translation.X;
    
    if (diff > maxDeltaSize)
    {
        diff = maxDeltaSize;
    }
    else if (diff < 0)
    {
        diff = 0;
    }
    
    coverRectangle->Width = initialBarWidth + diff;
}

When the user action on the button is completed, the ManipulationCompleted event handler is triggered on the same overlay grid. If the relative drag hits the critical threshold, the action linked to the button should be invoked:

void MenuItem::Grid_ManipulationCompleted(Platform::Object^ sender, Windows::UI::Xaml::Input::ManipulationCompletedRoutedEventArgs^ e)
{
    float diff = e->Cumulative.Translation.X;
    
    if (diff > maxDeltaSize - 50) // slight buffer
    {
        slideMenuMedia->Play();
        OnButtonSelected(this, Label);
    }
    
    ResetElements();
}

There are also local sound effects that are played when the slider-button is tapped and moved to the end. Both are handled by separate MediaElement controls that avoid an internal sound file switch, calling instead the Play method as necessary. The OnButtonSelected event handler can be dynamically hooked in the application backend.

Each button also has a visible content area that can display a text label. It can be set through a DependencyProperty:

DependencyProperty^ MenuItem::_LabelProperty = 
    DependencyProperty::Register("Label", 
    Platform::String::typeid,
    MenuItem::typeid, 
    nullptr);

In its current configuration, the menu label can be also set in XAML:

<local:MenuItem x:Name="btnNewGame" Label="new game">
    <local:MenuItem.RenderTransform>
        <TranslateTransform></TranslateTransform>
    </local:MenuItem.RenderTransform>
</local:MenuItem>

When the button loses the focus, the state resets to the passive one, stopping the animation and resetting the arrow image to the single one:

void MenuItem::ResetElements()
{
    BitmapImage^ image = ref new BitmapImage(ref new Uri("ms-appx:///MenuItems/single_arrow.png"));
    MenuImage->Source = image;
    
    ((Storyboard^)ControlContainer->Resources->Lookup("ArrowAnimator"))->Stop();
    
    coverRectangle->Width = initialBarWidth;
    coverActiveRectangle->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
}

That’s about as complex as the menu item control will get. The menu container itself can be any Grid or StackPanel control. The way the menu items are used across the game states, there is no need to have a separate unified container.

The Main XAML Container & State Changes

Going back to DirectXPage.xaml, there are several parts of the XAML layout that should be highlighted. First and foremost, the game curtains—the static parts of the screen that are being displayed instead of blacking out parts of the viewport that are not being used when the game runs in landscape mode. Because the game includes a rectangular frame instead of stretching the entire playable area to the size of the screen, there is an unknown amount of unallocated visual space on both the right and left sides of the frame itself:

<Grid HorizontalAlignment="Left" x:Name="containerA">
    <Rectangle Fill="#09bbe3" />

    <Rectangle Width="10" HorizontalAlignment="Left">
        <Rectangle.Fill>
            <LinearGradientBrush StartPoint="0,0" EndPoint="1,0">
                <GradientStop Color="#a000" Offset="0.0"></GradientStop>
                <GradientStop Color="#0000" Offset="1.0"></GradientStop>
            </LinearGradientBrush>
        </Rectangle.Fill>
    </Rectangle>
</Grid>

<Grid HorizontalAlignment="Right" x:Name="containerB">
    <Rectangle Fill="#09bbe3" />

    <Rectangle Width="10" HorizontalAlignment="Right">
        <Rectangle.Fill>
            <LinearGradientBrush StartPoint="1,0" EndPoint="0,0">
                <GradientStop Color="#a000" Offset="0.0"></GradientStop>
                <GradientStop Color="#0000" Offset="1.0"></GradientStop>
            </LinearGradientBrush>
        </Rectangle.Fill>
    </Rectangle>
</Grid>

Although these two grids have set alignments, the actual location on the screen will be set in code-behind because the size will also be re-calculated. Also, I need to make sure that the application is in the full display mode—if it is snapped, there is no need to display the curtains because the visible area will be reduced to an overlay grid.

The curtain resize and visibility are determined in the UpdateWindowSize method:

void DirectXPage::UpdateWindowSize()
{
    bool visibility = true;
    if (ApplicationView::Value == ApplicationViewState::Snapped)
            visibility = false;
    
    float margin = (m_renderer->m_renderTargetSize.Width - 768.0f) / 2.0f;
    if (margin < 2.0)
            visibility = false;
    
    if (visibility)
    {
        containerA->Width =  margin;
        containerA->HorizontalAlignment = Windows::UI::Xaml::HorizontalAlignment::Right;
        containerB->Width =  margin;
        containerB->HorizontalAlignment = Windows::UI::Xaml::HorizontalAlignment::Left;
        containerA->Visibility = Windows::UI::Xaml::Visibility::Visible;
        containerB->Visibility = Windows::UI::Xaml::Visibility::Visible;
    }
    else
    {
        containerA->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
        containerB->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
    }
}

The snippet above ensures that the curtains will only be displayed when there is extra, unused space in addition to the 768 pixels taken by the playable area.

Let’s take a look at state-specific content that is being displayed whenever the game switches states. Because of the nature of DirectX interaction, there is no way for me to hook to specific event handlers from the native loop. Therefore, I need to constantly check that the content displayed is associated with the current state.

This can be done with the help of the SwitchGameState method:

void DirectXPage::SwitchGameState()
{
    switch (m_renderer->CurrentGameState)
            {
            case GameState::GS_FULL_WIN:
                    {
                            grdCompleteWin->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_PLAYING:
                    {
                            Hud->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_MAIN_MENU:
                    {
                            stkMainMenu->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_GAME_OVER:
                    {
                            UpdateResults();
                            ResultPanel->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            grdGameOver->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_SUBMIT_SCORE:
                    {
                            grdSubmitScore->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            txtSubmitScoreView->Text = StaticDataHelper::Score.ToString();
                            break;
                    }
            case GameState::GS_TOP_SCORES:
                    {
                            grdTopScores->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_WIN:
                    {
                            UpdateResults();
                            grdWinner->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            ResultPanel->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_ABOUT_SCREEN:
                    {
                            grdAbout->Visibility =  Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_LEVEL_SELECT_SINGLE:
                    {
                            animationBeginTime = 0;
            
                            stkLevelSelector->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_HOW_TO:
                    {
                            grdHowTo->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            Storyboard^ howToInitialBoard = (Storyboard^)grdMain->Resources->Lookup("StoryboardChainA");
                            howToInitialBoard->Begin();
                            break;
                    }
        default:
                    break;
            }
    
            Storyboard^ loc = (Storyboard^)grdMain->Resources->Lookup("FadingOut");
            loc->Begin();
}

One thing you’ve probably noticed about the snippet above is the fact that there is no indicator showing that controls are being hidden when the state changes. Just calling SwitchGameState in the Update loop would cause multiple controls to be displayed at once. However, there is also the HideEverything method that goes through the visual tree and sets the Visibility to Collapsed for everything:

void DirectXPage::HideEverything()
{
    for (uint i = 0; i < grdMain->Children->Size; i++)
    {
        grdMain->Children->GetAt(i)->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
    }
}

HUD Interaction

The Heads-Up Display (HUD) is used to alert the player about the current state of the game and the game character. In FallFury, it is used to display three indicators: how many buttons the character collected, how much time it took the character to get to the current level part, and current character health. It is also the container for the Pause button.

Here is the visual representation:

Here is the underlying XAML:

<Grid x:Name="Hud" VerticalAlignment="Top" Visibility="Collapsed">
    <Grid.RowDefinitions>
        <RowDefinition Height="80" />
        <RowDefinition Height="Auto" />
    </Grid.RowDefinitions>
    <Grid.ColumnDefinitions>
        <ColumnDefinition />
        <ColumnDefinition />
        <ColumnDefinition />
        <ColumnDefinition Width="80" />
    </Grid.ColumnDefinitions>

    <Rectangle Fill="Black" Grid.ColumnSpan="4" />

    <Button
        Grid.Column="0"
        x:Name="btnPause"
        Click="btnPause_Click"
        Style="{StaticResource PauseButton}">
        <Image Source="ms-appx:///Assets/HUD/pauseButton.png" Stretch="None" /
    </Button>

    <StackPanel Grid.Column="1" Orientation="Horizontal">
        <Image Source="ms-appx:///Assets/HUD/buttonHud.png" Stretch="None" />

        <TextBlock
            x:Name="txtButtons"
            Text="0"
            Style="{StaticResource hudResult}"/>
    </StackPanel>

    <StackPanel Grid.Column="2" Orientation="Horizontal">
        <Image Source="ms-appx:///Assets/HUD/clockHud.png" Stretch="None" />

        <TextBlock
            x:Name="txtTimer"
            Text="00:00"
            Style="{StaticResource hudResult}"/>
    </StackPanel>

    <Image
        Grid.Column="3"
        Source="ms-appx:///Assets/HUD/heartHud.png"
        Stretch="None" />

    <Grid
        Grid.Column="3"
        Grid.Row="1">
        <Rectangle Fill="Black" Opacity=".8"/>
        <controls:HealthBar x:Name="healthBar"></controls:HealthBar>
    </Grid>
</Grid>

As I mentioned before, the DirectX layer cannot directly interact with the XAML layer. Therefore, there needs to be an intermediary binding class. As the game progress changes, the UpdateHUD method is called, taking a reference to the current DirectX screen and reading the game data from the character and the StaticDataHelper class, which is the container for the time elapsed:

void DirectXPage::UpdateHud(GamePlayScreen^ playScreen)
{
    healthBar->Update(playScreen->GameBear->CurrentHealth, playScreen->GameBear->MaxHealth);
    
    txtButtons->Text = StaticDataHelper::ButtonsCollected.ToString();
    
    // Find a better built-in string formatting code
    if (m_renderer->CurrentGameState == GameState::GS_PLAYING && !(StaticDataHelper::IsPaused) && playScreen->IsLevelLoaded)
    {
        StaticDataHelper::SecondsTotal += m_timer->Delta;
        
        txtTimer->Text = StaticDataHelper::GetTimeString((int)StaticDataHelper::SecondsTotal);
    }
}

As seen above, part of the HUD is taken by a health indicator control—HealthBar. It is a simple composite element made out of two overlaying rectangles, one of which is resized as the bear health changes:

<UserControl
    x:Class="Coding4Fun.FallFury.Controls.HealthBar"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:FallFury"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    d:DesignHeight="410"
    d:DesignWidth="20" Height="410" Width="20">
    
    <Grid>
        <StackPanel VerticalAlignment="Top" Margin="0, 5, 0,20">
            <Rectangle Height="400" Style="{StaticResource HealthRectangle}" />
        </StackPanel>

        <StackPanel VerticalAlignment="Top" Margin="0, 5, 0, 10">
            <Rectangle x:Name="OverlayStacker" Height="0" Style="{StaticResource HealthOverlayRectangle}" />
        </StackPanel>
    </Grid>
</UserControl>

Being of a fixed size, it is fairly easy to calculate the health-to-damage ratio and display that as the size of the overlaid rectangle:

DependencyProperty^ HealthBar::_MaxHealthProperty = 
    DependencyProperty::Register("MaxHealth", 
    double::typeid,
    HealthBar::typeid, 
    nullptr);

DependencyProperty^ HealthBar::_CurrentHealthProperty = 
    DependencyProperty::Register("CurrentHealth", 
    double::typeid,
    HealthBar::typeid, 
    nullptr);

void HealthBar::Update(double currentHealth, double maxHealth)
{
    CurrentHealth = currentHealth;
    MaxHealth = maxHealth;
    
    if (CurrentHealth >= 0)
    {
        OverlayStacker->Height = 400.0 - ((400.0 * CurrentHealth) / MaxHealth);
    }
}

Conclusion

Using XAML as a part of a DirectX application does not require the developer to do a massive overhaul of the infrastructure. That said, be mindful when deciding whether to use a hybrid XAML application, as it is much easier to integrate a SwapChainBackgroundPanel with the underlying DirectX configuration from the ground up instead of trying to do so when the DirectX component is completed.

The Background

Each game screen in FallFury has a moving background that creates the illusion of a fall. The way the screen is designed, it simulates vertical parallax scrolling, as the background moves faster than the overlaid objects. There is a simple way to make background movement possible without actually having to replicate parts of the texture and render them all over again. Take a look at this image, showcasing the process:

First of all, two textures are loaded that, connected at the bottom, create the illusion of a single texture. In the beginning, texture A takes the entire screen and texture B is positioned directly underneath it, with a non-existing gap between them. To initiate the scrolling, texture A is being displaced vertically by an arbitrary number of pixels, and texture B follows it at the same pace. As texture B reaches the zero point (top of the viewport), texture A is no longer visible, therefore it is displaced vertically to be below texture B. This cycle can be repeated as many times as necessary during gameplay as well as while the user is in the menu.

Let’s take a look at the code that makes it possible, starting with the main menu screen:

First of all, you need to be aware that every game screen is automatically inheriting the properties and capabilities of base class GameScreenBase. This is the class the offers the foundation both for basic texture loading and movement:

protected private:
    // Core background textures
    Microsoft::WRL::ComPtr<ID3D11Texture2D>                    m_backgroundBlockA;
    Microsoft::WRL::ComPtr<ID3D11Texture2D>                    m_backgroundBlockB;

void MoveBackground(float velocity);

It also offers the core texture containers for the overlaid elements that are present in most screens, such as clouds:

// Overlayed clouds
Microsoft::WRL::ComPtr<ID3D11Texture2D> m_overlayA;
Microsoft::WRL::ComPtr<ID3D11Texture2D> m_overlayB;

The overlay movement is inherently dependent on the base background displacement and can be adjusted relative to the initial velocity. Let’s take a look at GameScreenBase.cpp, specifically at MoveBackground:

void GameScreenBase::MoveBackground(float velocity)
{
    if (m_backgroundPositionA <= -BACKGROUND_MIDPOINT)
        m_backgroundPositionA = m_backgroundPositionB + (BACKGROUND_MIDPOINT * 2);

    if (m_backgroundPositionB <= -BACKGROUND_MIDPOINT)
        m_backgroundPositionB = m_backgroundPositionA + (BACKGROUND_MIDPOINT * 2);

    m_backgroundPositionA -= velocity;
    m_backgroundPositionB -= velocity;
}

The BACKGROUND_MIDPOINT value is relative to the height of the background texture. As we are working with a variable screen size, given that tablets and desktops do not have the same resolution, the movement has to be adjusted accordingly. One way, however, to ensure the proper texture positioning would be to place it in relation to its previous instance. Hence, this snippet in UpdateWindowSize:

BACKGROUND_MIDPOINT = 1366.0f / 2.0f;
m_backgroundPositionA = BACKGROUND_MIDPOINT;
m_backgroundPositionB = m_backgroundPositionA * 3;

Because overlays are not necessarily a part of every screen, I am not including the MoveOverlay method in the base class. Let’s now take a look at MenuScreen.cpp. Take a look at the Load method and you will see several lines that prepare the background and overlays:

m_loader = ref new BasicLoader(Manager->m_d3dDevice.Get(), Manager->m_wicFactory.Get());

m_loader->LoadTexture("Assets\\Backgrounds\\generic_blue_a.png", &m_backgroundBlockA, nullptr);
m_loader->LoadTexture("Assets\\Backgrounds\\generic_blue_b.png", &m_backgroundBlockB, nullptr);
m_loader->LoadTexture("Assets\\Backgrounds\\cloud_overlay_a.png", &m_overlayA, nullptr);
m_loader->LoadTexture("Assets\\Backgrounds\\cloud_overlay_b.png", &m_overlayB, nullptr);

CurrentSpriteBatch->AddTexture(m_backgroundBlockA.Get());
CurrentSpriteBatch->AddTexture(m_backgroundBlockB.Get());
CurrentSpriteBatch->AddTexture(m_overlayA.Get());
CurrentSpriteBatch->AddTexture(m_overlayB.Get());

The BasicLoader class was imported from the Direct3D sprite sample. A call to LoadTexture will read the data from a DDS or PNG file and output the data in an ID3D11Texture2D object. Once loaded, the texture is added to the SpriteBatch instance associated with the screen, also declared as a part of GameScreenBase.

Depending on the screen, this procedure might have to be done for multiple textures, as you will see later in this article. Each page also has two timed loops—RenderScreen and Update. RenderScreen is responsible for taking everything from the SpriteBatch instance and showing it to the user. If you’ve used the XNA SpriteBatch before, you are aware that you need to start the drawing cycle by calling Begin and end it by calling End. The same applies here:

CurrentSpriteBatch->Begin();

CurrentSpriteBatch->Draw(
    m_backgroundBlockA.Get(),
    float2(Manager->m_windowBounds.Width / 2, m_backgroundPositionA),
    PositionUnits::DIPs,
    m_screenSize,
    SizeUnits::Pixels);

CurrentSpriteBatch->Draw(
    m_backgroundBlockB.Get(),
    float2(Manager->m_windowBounds.Width / 2 ,m_backgroundPositionB),
    PositionUnits::DIPs,
    m_screenSize,
    SizeUnits::Pixels);

if (m_showBear->IsLoaded)
    m_showBear->Render();

if (m_showMonster->IsLoaded)
    m_showMonster->Render();

CurrentSpriteBatch->Draw(
    m_overlayA.Get(),
    float2(Manager->m_windowBounds.Width/2, m_backgroundPositionA),
    PositionUnits::DIPs,
    m_screenSize,
    SizeUnits::Pixels);

CurrentSpriteBatch->Draw(
    m_overlayB.Get(),
    float2(Manager->m_windowBounds.Width / 2 ,m_backgroundPositionB),
    PositionUnits::DIPs,
    m_screenSize,
    SizeUnits::Pixels);

CurrentSpriteBatch->End();

The current overloaded Draw call gets the following items:

• The texture object
• The position where the object has to be drawn on the screen
• The way the object is positioned (can also be a normalized value or a pixel value)
• The size of the texture (stretching may occur)
• The type of the size value (in this case, I am using pixels, but can also be normalized)

The order in which the Draw calls are arranged determines the order of objects drawn on the screen. The calls on top will place texture objects at the bottom of the rendering stack, and the calls at the end will place the objects at the top of the stack.

The Gameplay Screen Background & Overlays

Let’s move on to the arguably most critical screen in the project, GamePlayScreen.cpp. There are a few nuances that differentiate it from any other screen—in particular, the background loading routine.

For every game, there is a different level type that is being used. With each level type, there is a different combination of a background and the overlay. Currently, there are four supported level types:

• Space
• Nightmare
• Magic Bean
• Dream

When the proper level is detected and the metadata is loaded from the associated XML file (more about this process later on in the series), the level textures are loaded into memory:

switch (m_currentLevelType)
{
    case LevelType::SPACE:
    {
        m_loader->LoadTexture("Assets\\Backgrounds\\generic_dark_blue_a.png", &m_backgroundBlockA, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\generic_dark_blue_b.png", &m_backgroundBlockB, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\star_overlay_a.png", &m_overlayA, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\star_overlay_b.png", &m_overlayB, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\galaxy_overlay_a.png", &m_overlayGalaxyA, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\galaxy_overlay_b.png", &m_overlayGalaxyB, nullptr);
        
        CurrentSpriteBatch->AddTexture(m_overlayGalaxyA.Get());
        CurrentSpriteBatch->AddTexture(m_overlayGalaxyB.Get());
        break;
    }
    case LevelType::NIGHTMARE:
    {
        m_loader->LoadTexture("Assets\\Backgrounds\\generic_red_a.png", &m_backgroundBlockA, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\generic_red_b.png", &m_backgroundBlockB, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\cloud_overlay_a.png", &m_overlayA, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\cloud_overlay_b.png", &m_overlayB, nullptr);
        break;
    }
    case LevelType::MAGIC_BEANS:
    {
        m_loader->LoadTexture("Assets\\Backgrounds\\generic_blue_a.png", &m_backgroundBlockA, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\generic_blue_b.png", &m_backgroundBlockB, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\cloud_overlay_a.png", &m_overlayA, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\cloud_overlay_b.png", &m_overlayB, nullptr);
        
        break;
    }
    case LevelType::DREAM:
    {
        m_loader->LoadTexture("DDS\\Levels\\Dream\\TEST_backgroundDream_01.dds", &m_backgroundBlockA, nullptr);
        m_loader->LoadTexture("DDS\\Levels\\Dream\\TEST_backgroundDream_02.dds", &m_backgroundBlockB, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\cloud_overlay_a.png", &m_overlayA, nullptr);
        m_loader->LoadTexture("Assets\\Backgrounds\\cloud_overlay_b.png", &m_overlayB, nullptr);
        break;
    }
}

This way, no unnecessary textures are loaded. A screen-based level flag allows me to control the incoming assets. At this point, the level environment does not change dynamically, so it is safe to assume that the textures should be loaded on a per-level basis.

The RenderScreen method is called in the same manner as in the menu screen, with the moving background and overlays located at the bottom of the rendering stack:

CurrentSpriteBatch->Draw(
    m_backgroundBlockA.Get(),
    float2(Manager->m_windowBounds.Width / 2, m_backgroundPositionA),
    PositionUnits::DIPs,
    m_screenSize,
    SizeUnits::Pixels);

CurrentSpriteBatch->Draw(
    m_backgroundBlockB.Get(),
    float2(Manager->m_windowBounds.Width / 2, m_backgroundPositionB),
    PositionUnits::DIPs,
    m_screenSize,
    SizeUnits::Pixels);

CurrentSpriteBatch->Draw(
    m_overlayA.Get(),
    float2(Manager->m_windowBounds.Width / 2, m_overlayPositionA),
    PositionUnits::DIPs,
    float2(768.0f, 1366.0f),
    SizeUnits::Pixels);

CurrentSpriteBatch->Draw(
    m_overlayB.Get(),
    float2(Manager->m_windowBounds.Width / 2, m_overlayPositionB),
    PositionUnits::DIPs,
    float2(768.0f, 1366.0f),
    SizeUnits::Pixels);

Character Movement

As you are now aware of the basic screen structure and how the basic rendering process is built, let’s take a look at how the main game character moves on the screen. Falling down, the teddy bear also needs to move left and right to ensure that he is able to pick up power-ups and buttons as well as avoid obstacles and enemy ammo.

There are several important considerations here. The most important one is to not assume that the user will have a specific input device. Potential movement controllers include a keyboard, the mouse, the touch screen and the accelerometer. In the most common scenarios, the desktop machines will not have an accelerometer, and the tablet computers will not have a constantly attached keyboard and a mouse. In FallFury, I decided to leverage all potential input engines and let the user choose the best option for himself.

When the current GamePlayScreen instance loads, I am attempting to get access to the system accelerometer device:

m_systemAccelerometer = Windows::Devices::Sensors::Accelerometer::GetDefault();

m_systemAccelerometer is of type Windows::Devices::Sensors::Accelerometer and is declared in GamePlayScreen.h. If an accelerometer is detected, I need to bind it to a ReadingChanged event handler that will give me the current G-force transformed into X, Y, and Z displacement:

if (m_systemAccelerometer != nullptr)
{
    m_systemAccelerometer->ReadingChanged += ref new TypedEventHandler<Accelerometer^, AccelerometerReadingChangedEventArgs^>
            (this, &GamePlayScreen::AccelerometerReadingChanged);
}

ReadingChanged itself does not participate in updating the position for the character, but rather passes the current values to m_xAcceleration, which is later used in the Update method:

void GamePlayScreen::AccelerometerReadingChanged(_In_ Accelerometer^ accelerometer, _In_ AccelerometerReadingChangedEventArgs^ args)
{
    if (StaticDataHelper::IsAccelerometerEnabled)
    {
        auto currentOrientation = DisplayProperties::CurrentOrientation;
        float accelValue;
        
        if (currentOrientation == DisplayOrientations::Portrait)
                    accelValue = args->Reading->AccelerationY;
        else if (currentOrientation == DisplayOrientations::PortraitFlipped)
                    accelValue = -args->Reading->AccelerationY;
        else if (currentOrientation == DisplayOrientations::Landscape)
                    accelValue = args->Reading->AccelerationX;
        else if (currentOrientation == DisplayOrientations::LandscapeFlipped)
                    accelValue = -args->Reading->AccelerationX;
        else
                    accelValue = 0.0f;
        
        if (StaticDataHelper::IsAccelerometerInverted)
                    m_xAcceleration = -accelValue;
        else
                    m_xAcceleration = accelValue;
    }
}

The reason for this lies in the fact that ReadingChanged is triggered at a much lower rate than the Update loop. If the character position would be adjusted through the core accelerometer event handler, the result would be choppy (“step-by-step”) movement instead of a smooth transition.

Notice, also, that depending on the screen orientation, I need to get the acceleration either on the X- or Y-axes. Since an accelerometer is detected, I am assuming that the device that’s being used is a tablet. Therefore, it can have auto-rotate enabled, which means that the reference axis (horizontal) might change depending on how the user holds the device. DisplayProperties::CurrentOrientation can give me the current orientation, whether portrait (Y acceleration value) or landscape (X acceleration value):

As with many other titles, the user may invert the movement based on the accelerometer reading. For example, if the device is tilted to the right, the character will move to the left, and vice-versa. The effect is easily achieved by negating the current reading, regardless of the axis it is relying on.

Obviously, as the character moves, there should be boundaries that restrict the movement within the context of the current game screen. To get the correct screen bounds, GameScreenBase, the foundation class for every screen, sets four properties: LoBoundX, HiBoundX, LoBoundY and HiBoundY:

LoBoundX = (rWidth - 768.0f) / 2.0f;
HiBoundX = LoBoundX + 768.0f;

LoBoundY = 0;
HiBoundY = LoBoundY + rHeight;

LoBoundX is the leftmost limit for the horizontal playable area and HiBoundX is the rightmost limit for the same horizontal area. LoBoundY and HiBoundY are responsible for carrying the limits for the vertical space. All four boundary values are relative to the screen size:

To consistently get the correct X boundary no matter the screen size, the playable area is set to a fixed width of 768 pixels. The rest of the screen is accordingly cancelled out and the remaining area split in two. The original X point (zero) is added to the curtain size and that way there is the leftmost X boundary. The Y boundaries are simply obtained from the screen height, as there are no gameplay experience restrictions in that domain.

The bear position is updated in the Update loop in the game screen. There are several parameters that need to be adjusted, such as the vertical velocity, the bear rotation on tilt and the horizontal displacement.

When the level just starts, the bear falls faster until he reaches the standard static Y point. Because of screen size differences, this should not be done by comparing the pixel distance but rather by utilizing a ratio built from the current bear Y position and the high Y boundary:

if ((GameBear->Position.y / HiBoundY) < 0.19f)
{
    GameBear->Position.y += GameBear->Velocity.y * (3.2f);
}

Assuming that the game is not paused, and thus the background is moving, the bear Y position should be adjusted relative to the current velocity set for the background scrolling. Because of how parallax scrolling works, the bear velocity has to be higher than the initial screen movement. Therefore, it is multiplied by a fixed value independent of the level played:

if (!m_isBackgroundMoving)
{
    if (GameBear->Position.y > m_screenSize.y)
    {
        Manager->CurrentGameState = GameState::GS_GAME_OVER;
    }
    else
    {
        GameBear->Position.y += GameBear->Velocity.y * 1.5f;
    }
}

In the statement above, there is also an extra condition that verifies whether the bear is below the low Y boundary. If it is, then the game is over. This is imposed by a game animation that takes the bear off the Y limits, meaning that the bear is dead.

Bear rotation is based on the current X acceleration, but the bear shouldn’t rotate 360 degrees. To limit the rotation, there is a rotation threshold set (value taken in radians) that is being checked before rotating the character:

float compositeRotation = GameBear->Rotation - (float)m_xAcceleration / 10.0f;
if (compositeRotation < m_rotationThreshold && compositeRotation > -m_rotationThreshold)
    GameBear->Rotation = compositeRotation;

The composite value lets me verify the perspective rotation without actually assigning it to the bear. If it is within the imposed threshold, only then will the bear rotation be adjusted.

Using the composite value approach is also efficient when performing the horizontal displacement adjustment. The initial value is composed of the current position and the X-based acceleration multiplied by a dynamic multiplier value to create the inertia effect:

float compositePosition = GameBear->Position.x + ((float)m_xAcceleration * m_accelerationMultiplier);

if (m_xAcceleration < 0)
    compositePosition -= 100.0f;
else
    compositePosition += 100.0f;

if (Manager->IsWithinScreenBoundaries(float2(compositePosition, GameBear->Position.y)))
{
    GameBear->Position.x += (float)m_xAcceleration * m_accelerationMultiplier;
}
else
{
    if (GameBear->Position.x > HiBoundX)
    {
        GameBear->Position.x = HiBoundX - 180.0f;
    }
    else if (GameBear->Position.x < LoBoundX)
    {
        GameBear->Position.x = LoBoundX + 180.0f;
    }
}

If the bear is attempting to hit a “wall” (screen limit), its position is set back to the one barely prior to the limit. By doing this, I am avoiding locking the character on the side of the screen.

There is something in this snippet above that you might not be aware of—a reference to Manager->IsWithinScreenBoundaries. This method belongs to the ScreenManager (ScreenManager.cpp) class—a utility class that ensures the proper screen is displayed depending on the current game mode, and also allows control of mouse actions and boundary checks:

bool ScreenManager::IsWithinScreenBoundaries(float2 position)
{
    if (position.x < CurrentGameScreen->LoBoundX || position.x > CurrentGameScreen->HiBoundX || position.y < CurrentGameScreen->LoBoundY || position.y > CurrentGameScreen->HiBoundY)
            return false;
    else
            return true;
}

You saw how the movement is accomplished with the help of the accelerometer. Let’s take a look at how a keyboard can be used to do the same thing. There is a HandleKeyInput method, exposed through the GamePlayScreen class. As a matter of fact, HandleKeyInput is wired into the GameScreenBase class, therefore if a specific combination needs to be handled outside the context of the game play screen, it can be:

void GamePlayScreen::HandleKeyInput(Windows::System::VirtualKey key)
{
    if (key == Windows::System::VirtualKey::Right)
    {
        if (GameBear->IsWithinScreenBoundaries(GameBear->Size.x, 0.0f, GetScreenBounds()))
        {
            GameBear->Direction = TurningState::RIGHT;
            
            m_xAcceleration = 0.8f;
            
            if (GameBear->Rotation >= -m_rotationThreshold)
                            GameBear->Rotation -= 0.02f;
        }
    }
    else if (key == Windows::System::VirtualKey::Left)
    {
        if (GameBear->IsWithinScreenBoundaries(-GameBear->Size.x, 0.0f, GetScreenBounds()))
        {
            GameBear->Direction = TurningState::LEFT;
            
            m_xAcceleration = -0.8f;
            
            if (GameBear->Rotation <= m_rotationThreshold)
                            GameBear->Rotation += 0.02f;
        }
    }
}

As the accelerometer is no longer influencing the rotation or displacement, both indicators have to be manipulated through key presses. There are still standard thresholds in place to limit the potential incorrect movement, but the idea remains the same. This method is being called from the XAML page, which is overlaid on top of the DirectX renders:

<SwapChainBackgroundPanel
    x:Class="Coding4Fun.FallFury.DirectXPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:Coding4Fun.FallFury" x:Name="XAMLPage"
    xmlns:controls="using:Coding4Fun.FallFury.Controls"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d" Loaded="OnLoaded" KeyDown="OnKeyDown" LayoutUpdated="XAMLPage_LayoutUpdated">

void DirectXPage::OnKeyDown(Platform::Object^ sender, Windows::UI::Xaml::Input::KeyRoutedEventArgs^ e)
{
    m_renderer->CurrentGameScreen->HandleKeyInput(e->Key);
}

When the keyboard is not available, movement can be controlled with the help of a mouse. Again, with standard event handlers, I am simply using OnPointerMoved:

void GamePlayScreen::OnPointerMoved(Windows::UI::Core::PointerEventArgs^ args)
{
    if (StaticDataHelper::IsMouseEnabled)
    {
        if (GameBear != nullptr)
        {
            m_touchCounter++;
            if (GameBear->IsWithinScreenBoundaries(GameBear->Size.x, 0.0f, GetScreenBounds()))
            {
                m_xAcceleration = (args->CurrentPoint->RawPosition.X - GameBear->Position.x) / m_screenSize.x;
            }
        }
    }
}

By using this method, the bear will accelerate to the point where the mouse cursor is located, having a higher velocity the further it is located from the cursor. Once again, this creates the inertia visualization.

Conclusion

Concluding Part 3 of the series, remember that because Windows 8 is not a tablet-only OS, some users might have different input devices. Try to accommodate as many of those as possible.

The next article in this series will focus on the XAML overlay used in FallFury.

FallFury not only relies on standard C++/C# and XAML code, but also on shaders. This article is intended for developers who are not aware of what shaders are and want to know how to use them in their projects. I will talk about creating shaders, as well as the shaders used in my project.

Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-2-Shaders.  For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

Code on GPU

In simple terms, shaders are small programs that are executed on the Graphical Processing Unit (GPU) instead of the Central Processing Unit (CPU). In recent years, we’ve seen a major spike in the capabilities of graphic devices, allowing hardware manufacturers to design an execution layer tied to the GPU, therefore being able to target device-specific manipulations to a highly optimized unit. Shaders are not used for simple calculations, but rather for image processing. For example, a shader can be used to adjust image lighting or colors.

Modern GPUs give access to the rendering pipeline that allows developers to execute arbitrary image processing code. This is a step forward from a fixed-function pipeline that was present in older GPUs, where image processing tasks were integrated into the hardware unit and were only able to perform a limited set of actions, such as transforms. Unlike the rendering pipeline, the fixed-function pipeline was not programmable, therefore the developers were often tied to the hardware they used for offering specific game effects, in some cases having to resort to software-based adjustments.

Types of Shaders

There are different types of image manipulations that can be performed on a given input, and so there are different types of shaders designed to handle that. Currently, we can highlight three main shader types:

• Vertex shaders – because what the user sees on the screen is not really three-dimensional, but rather a three-dimensional simulation in a 2D space, vertex shaders translate the coordinates of a vector in 3D space in relation to the 2D frame. Vertex shaders are executed one time per vector passed to the GPU. Typically, vectors carry data related to their position and the coordinate of the bound texture as well as color. A vertex shader is able to manipulate all these properties, but there is never a situation where a new vertex is created as a result of the execution.
• Pixel shaders – these programs are executed on the GPU in relation to every passed pixel, working on a much lower level. For example, if you want specific pixels adjusted for lighting or 3D bump mapping, a pixel shader can provide the desired effect for a surface. Rarely, there are situations where only a few pixels should be re-adjusted at once. Pixel shaders are often run with an input of millions of pixels, resulting in complex effects.
• Geometry shaders – these shaders are the next progression from vertex shaders, introduced with DirectX 10. The developer is able to pass specific primitives as input and either have the output represent the modified version of what was passed to the program or have new primitives, such as triangles, be generated as a result. Geometry shaders are always executed on post-vertex processing in the rendering pipeline. When vertex shader execution is completed, geometry shaders step in, if present. Geometry shaders can be used to refine the level of detail of a specific object. For example, when an object is closer or further away from the viewport camera, the mesh would have to be refined to minimize the rendering load.

FallFury uses all these shader types as a part of the game.

Enough Talk, Let’s Code

Now that you are aware of what shaders are, let’s write some sample code and test it. It is worth mentioning that shaders are not written in a standard high-level language, but rather in a language defined by the environment the shader is used in. For Direct3D, this is the High-Level Shader Language, or HLSL. It is somewhat similar to C, but with some specific nuances.

Start by creating a test Direct3D Windows Store application:

If you create and run this sample application, you will see that the output of it is a simple 3D spinning cube:

You probably also noticed that there are two shaders that are a component part of the newly created solution: SimplePixelShader.hlsl and SimpleVertexShader.hlsl. Take a look inside the pixel shader:

struct PixelShaderInput
{
    float4 pos : SV_POSITION;
    float3 color : COLOR0;
};

float4 main(PixelShaderInput input) : SV_TARGET
{
    return float4(input.color,1.0f);
}

First of all, there is a PixelShaderInput struct[DW1] . It represents a single pixel – it has a float4 field that represents the position of the pixel and a float3 field that represents the RGB pixel color. The field itself is also marked by a predefined type, such as SV_POSITION or COLOR. This is a string that determines the use of the field. You can read more about the shader semantics here.

The strings that are prefixed with SV_ are representing system-value semantics. Those have special meaning in the pipeline during the processing stages. In the pixel shader above, SV_POSITION will always mean the pixel position.

Look at what is being returned from the pixel shader—instead of the standard float3 color indicator, you are now returning a float4, which couples the existing value, input.color, with a 1.0f float value that represents the alpha-channel value. Remember, that inside shaders the color is clamped between 0 and 1 instead of the standard 255-value limit.

Since the cube rendering mechanism is already in place, let’s experiment with the pixel shader a bit. You’ve already got the float4 color representation:

You can set any of these values to 1.0 to return the solid color, so let’s do that. Let’s render the cube green. Modify the return statement to be this:

return float4(0.0, 1.0, 0.0, 1.0);

At this point, if you will run the program you will get a rendering very similar to this:

This is a good start.. Now, let’s now take a look at the vertex shader in the project. By default, you get this:

cbuffer ModelViewProjectionConstantBuffer : register(b0)
{
    matrix model;
    matrix view;
    matrix projection;
};

struct VertexShaderInput
{
    float3 pos : POSITION;
    float3 color : COLOR0;
};

struct VertexShaderOutput
{
    float4 pos : SV_POSITION;
    float3 color : COLOR0;
};

VertexShaderOutput main(VertexShaderInput input)
{
    VertexShaderOutput output;
    float4 pos = float4(input.pos, 1.0f);
    
    pos = mul(pos, model);
    pos = mul(pos, view);
    pos = mul(pos, projection);
    output.pos = pos;
    
    // Pass through the color without modification.
    output.color = input.color;
    
    return output;
}

There are a couple of differences here compared to the above mentioned pixel shader:

• cbuffer ModelViewProjectionConstantBuffer – represents a constant buffer that contains three matrices, to which vertices are related—the model, view and projection matrices. Constant buffers are interesting structures that have been optimized to allow block-wise updates, where multiple constants are grouped in one boxed unit and can be updated simultaneously instead of having a one-by-one iterative update cycle.
• struct VertexShaderInput – vertices are passed one-by-one to the shader, and this specific structure represents the input. Notice that it carries the 3D representation of the vertex position as well as its RGB color value.
• struct VertexShaderOutput – the final processed output, which now carries the position in relation to all three matrices mentioned above as well as the processed color. Notice that the fourth value is the camera distance in the demo.
• VertexShaderOutput main(VertexShaderInput input) – this is the main function that performs the vertex processing. Initially, it performs the position adjustment relative to the camera and then uses mul, a built-in function that multiplies matrices, to represent the current vertices in the projected space.

So let’s play around with what we have in stock. Take a look at the main function and how initially, the float3-based position is transformed to contain the camera distance:

float4 pos = float4(input.pos, 1.0f);

This position is normalized in relation to the current viewport. Afterwards, the vertex is transformed in the current 3D space (relative to three matrices – model, view, and projection):

pos = mul(pos, model);
pos = mul(pos, view);

pos = mul(pos, projection);

output.pos = pos;

As with the pixel shader, these are some very basic manipulations. Let’s flip the cube upside-down. To do this, we need to apply a rotation transformation. Here is an interesting piece of advice—when working with manipulations in shaders, make sure that you know basic matrix operations.

For example, to perform a simple 2D rotation, you need to use the rotation matrix:

But since we’re in 3D space, we not only have the X and Y coordinates, which relate to the matrix above, but also the Z coordinate. Therefore, a different method should be applied. There are three fundamental matrices that can be used to rotate an object around the three possible axes:

For now, we want to rotate the cube along the X-axis. To do this, inside the main function, we need to declare a constant that represents the rotation angle, in radians:

const float angle = 1.3962634;

Looking at the matrix above, we need to find out the sine and cosine of the given angle. When working with vertex shaders, HLSL offers a built-in function, called sincos that is able to get both values from an input value:

float cosLength, sinLength;
sincos(angle, sinLength, cosLength);

Now you need to declare the rotation matrix. Again, HLSL comes with some built-in capabilities to declare matrices and set their values:

float3x3 xAxisRotation = { 
    1.0, 0.0, 0.0, // Row 1
    0, cosLength, -sinLength, // Row 2
    0, sinLength, cosLength}; // Row 3

The standard format for matrix declaration in this case follows the following pattern:

Matrix<type,size> localName

For matrix multiplication, we can once again leverage the mul function:

float3 temporaryPosition;
temporaryPosition = mul(input.pos,xAxisRotation);

Now we can normalize the position to a float4 value that also includes the camera distance:

float4 pos = float4(temporaryPosition, 1.0f);

The rest of the transformation procedures can be taken directly from the original shader, multiplying the float4 position by the model, view, and projection matrices. Your entire main function should now look like this:

VertexShaderOutput main(VertexShaderInput input)
{
    VertexShaderOutput output;

    const float angle = 1.3962634;
    float cosLength, sinLength;
    sincos(angle, sinLength, cosLength);

    float3x3 xAxisRotation = { 
        1.0, 0.0, 0.0, // Row 1
        0, cosLength, -sinLength, // Row 2
        0, sinLength, cosLength}; // Row 3

    float3 temporaryPosition;
    temporaryPosition = mul(input.pos,xAxisRotation);
    float4 pos = float4(temporaryPosition, 1.0f);

    // Transform the vertex position into projected space.
    pos = mul(pos, model);
    pos = mul(pos, view);
    pos = mul(pos, projection);
    output.pos = pos;

    // Pass through the color without modification.
    output.color = input.color;

    return output;
}

The current angle is set to 80 degrees, or 1.3962634 radians. If you run the rendering test application, the result you will get will be similar to this:

Now, let’s look at how shaders are handled on the DirectX side.

Shaders in DirectX

Open CubeRenderer.cpp. This is the source file where internal shaders are read and passed to the device to be executed. Notice one interesting aspect of the process – the shader file contents are being read first and then passed to m_d3dDevice->CreateVertexShader. m_d3dDevice is a pointer to a virtual adapter that can be used to create device-specific resources. CreateVertexShader creates a vertex shader from an already compiled shader. You can notice it from the fact that the data is read not from the initial .hlsl file, but rather from the .cso (Compiled Shader Object):

auto loadVSTask = DX::ReadDataAsync("SimpleVertexShader.cso");

As with any program, before it is passed onto the execution layer, it has to be compiled first. Visual Studio 2012 comes with a bundled HLSL compiler, fxc—the Effect Compiler Tool. The default behavior for a shader is to be compiled with the .cso file extension in the same output directory as the project, but this can be changed by setting the Object File Name in the shader properties:

You can read more about the shader compilation process here.

Looking back at the sample vertex shader that was created as a part of the project, you will notice that there is a specific input layout set for the incoming object:

struct VertexShaderInput
{
    float3 pos : POSITION;
    float3 color : COLOR0;
};

The virtual adapter is not aware of the structure layout. Therefore, the developer needs to explicitly create an internal D3D11_INPUT_ELEMENT_DESC array that will contain the type of information passed to the shader:

const D3D11_INPUT_ELEMENT_DESC vertexDesc[] = 
{
    { 
        "POSITION", 
        0, 
        DXGI_FORMAT_R32G32B32_FLOAT, 
        0, 
        0, 
        D3D11_INPUT_PER_VERTEX_DATA, 
        0 },
    {
        "COLOR", 
        0, 
        DXGI_FORMAT_R32G32B32_FLOAT, 
        0, 
        12, 
        D3D11_INPUT_PER_VERTEX_DATA, 
        0 },
};

Once the description is complete, the input needs to be assembled for processing. That’s where CreateInputLayout comes into play:

DX::ThrowIfFailed(
    m_d3dDevice->CreateInputLayout(
        vertexDesc,
        ARRAYSIZE(vertexDesc),
        fileData->Data,
        fileData->Length,
        &m_inputLayout
        )
    );

You are basically passing a shader signature to the input assembler that will perform all the consequent processing. A similar process is applied for the existing pixel shader, with the main difference being the fact that instead of CreateVertexShader, CreatePixelShader is called:

Passing Custom Parameters to Shaders

Going back to our vertex shader where we performed the rotation relative to the X-axis, you probably noticed that the angle is hard-coded and is applied to each vector in the same way. This is rarely the case, as the angle would normally come from inside the game itself, often responding to internal behavior, such as character movement or action.

To pass a parameter to the shader, you need to first of all redefine the input structure. Let’s add an angle carrier to the VertexShaderInput struct in SimpleVertexShader.hlsl:

struct VertexShaderInput
{
    float3 pos : POSITION;
    float3 color : COLOR0;
    float angle : TRANSFORM0;
};

This alone won’t do anything. Go the entry point function (main) and make sure that the angle constant is set to input.angle:

const float angle = input.angle;

Now the shader is ready, but you also need to let your game know that the vertex shader has a modified input layout. To do this, go to CubeRenderer.cpp and find the D3D11_INPUT_ELEMENT_DESC array that defines the input layout for incoming vertex shaders. Simply add an identifying item to the existing array:

const D3D11_INPUT_ELEMENT_DESC vertexDesc[] = 
{ 
    { "POSITION", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, 0, D3D11_INPUT_PER_VERTEX_DATA, 0 }, 
    { "COLOR", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, 12, D3D11_INPUT_PER_VERTEX_DATA, 0 }, 
    { "TRANSFORM", 0, DXGI_FORMAT_R32_FLOAT, 0, 24, D3D11_INPUT_PER_VERTEX_DATA, 0 } 
};

The first parameter is the pre-defined semantic used in the shader. The second parameter defines the input index. After that, use DXGI_FORMAT_R32_FLOAT to indicate that the value passed will be a standard float. Pay close attention when you specify the input offset. Notice that the two FLOAT3 values, POSITION and COLOR, take 12 bytes each —a float takes 32 bits (4 bytes), and you have a triplet, therefore the offset for TRANSFORM should be 24 bytes (two times 12 bytes from previous indicators).

Now you need to modify the internal vector descriptors that are being passed to the rendering pipe. In the sample project those are created with the help of the VertexPositionColor class, located in the CubeRenderer.h. Add a simple float field to the existing struct, so it looks like this:

struct VertexPositionColor 
{ 
    DirectX::XMFLOAT3 pos; 
    DirectX::XMFLOAT3 color; 
    float angle; 
};

In CubeRenderer.cpp, find the cubeVerticles array. For each created VertexPositionColor you are now able to add a third value representing the rotation:

auto createCubeTask = (createPSTask && createVSTask).then([this] () { 
    VertexPositionColor cubeVertices[] = 
    { 
        {XMFLOAT3(-0.5f, -0.5f, -0.5f), XMFLOAT3(0.0f, 0.0f, 0.0f), 1.38f}, 
        {XMFLOAT3(-0.5f, -0.5f, 0.5f), XMFLOAT3(0.0f, 0.0f, 1.0f), 1.38f}, 
        {XMFLOAT3(-0.5f, 0.5f, -0.5f), XMFLOAT3(0.0f, 1.0f, 0.0f), 1.38f}, 
        {XMFLOAT3(-0.5f, 0.5f, 0.5f), XMFLOAT3(0.0f, 1.0f, 1.0f), 1.38f}, 
        {XMFLOAT3( 0.5f, -0.5f, -0.5f), XMFLOAT3(1.0f, 0.0f, 0.0f), 1.38f}, 
        {XMFLOAT3( 0.5f, -0.5f, 0.5f), XMFLOAT3(1.0f, 0.0f, 1.0f), 1.38f}, 
        {XMFLOAT3( 0.5f, 0.5f, -0.5f), XMFLOAT3(1.0f, 1.0f, 0.0f), 1.38f}, 
        {XMFLOAT3( 0.5f, 0.5f, 0.5f), XMFLOAT3(1.0f, 1.0f, 1.0f), 1.38f}, 
    };

Now you will be able to pass parameters to the vertex shader without having to manually modify the shader itself.

Shaders in FallFury

There are several shaders used in FallFury, which must be selected depending on the Direct3D feature level available on the machine. A feature level determines what a video adapter can do in terms of rendering—even though Direct3D is a unified framework, it still heavily relies on how individual graphic cards can perform.

There are 5 shaders used internally for texture rendering. Specifically, there is a pixel shader, a replication vertex shader, an instancing vertex shader and geometry shaders. This could be familiar if you worked with Microsoft DirectX sprite samples before:

Given the platform limitations, geometry shaders can only be used on devices supporting Direct3D Feature Level 10. When FallFury detects that the feature level is lower than that, it has to fall back on either the instancing or the replication shaders. Some devices, such as the Microsoft Surface, are at feature level 9.1. Therefore, the replication render technique must be enforced.

Following the Microsoft DirectX Sprite Sample, it is fairly easy to simply select the correct render technique once the feature level is detected with GetFeatureLevel:

auto featureLevel = m_d3dDevice->GetFeatureLevel(); 
if (featureLevel >= D3D_FEATURE_LEVEL_10_0) 
{ 
    m_technique = RenderTechnique::GeometryShader; 
} 
else if (featureLevel >= D3D_FEATURE_LEVEL_9_3) 
{ 
    m_technique = RenderTechnique::Instancing; 
} 
else 
{ 
    m_technique = RenderTechnique::Replication; 
    if (capacity > static_cast<int>(Parameters::MaximumCapacityCompatible)) 
    { 
        // The index buffer format for feature-level 9.1 devices may only be 16 bits. 
        // With 4 vertices per sprite, this allows a maximum of (1 << 16) / 4 sprites. 
        throw ref new Platform::InvalidArgumentException(); 
    } 
}

Depending on the selected render technique, which is determined on application startup, the proper input layout is selected for each shader and is used to control the rendering pipe.

Different graphic adapters also support different shader models. You need to account for those, and before running the application, the HLSL compiler will take on the task of determining whether a proper shader component is supported by the given model. In the shader properties, make sure that you specify the used shader model, as well as the type of the shader:

Without this, you are bound to experience compile time errors that will prevent you from launching the application or deploying it to different devices.

Conclusion

Shaders are not an easy subject. I highly recommend reading The Cg Tutorial, published online for free by nVidia. Even though it describes the Cg shader language, it is virtually identical to HLSL and you will not have any problems using the practices you learned there in building HLSL shaders.

FallFury is a 2D platformer in which the player controls a falling bear, trying to avoid obstacles, dodge missiles, and destroy monsters as the bear falls. The project incorporates several of the new Windows 8 APIs, including the accelerometer and touch as well as integrations with core OS capabilities such as settings and share charms. Additionally, the project leverages the most exacting addition to the Visual Studio development environment—hybrid application development with XAML, C++, and DirectX.

Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-1-Introduction.  For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

Design & Idea

From the outset, Rick Barraza and I decided that since our target audience was composed of both kids and adults, the main character had to be familiar to both groups. Teddy bears turned out to be the best choice. Rick spent a day creating tens of potential bear drawings—out of which I had to choose one:

As the bear’s fall progresses, the character encounters a variety of obstacles dependent on the level type and complexity. Those obstacles should of course be avoided, so as the user tilts the device, the character in the game moves in the associated direction.

The design of the project took a week, and during this time the following items were determined and conceptualized. Considerations for both tablet and desktop environments directed our decisions:

· The main character layout.

· The way the game progresses as the character falls down.

· How the user interacts with the game in a wide variety of possible scenarios.

· What the game screens look like.

· What the menu system interaction looks like.

· How the game looks in different screen modes and on different device types.

· Some of the bonuses that the main character can pick up during free fall.

· What happens when the user progresses through the game and iterates through levels.

· What is shared and how this is accomplished.

As the game ideas were outlined, Arturo Toledo, the designer behind ux.artu.tv, was brought on to create the game assets. Then the core design decisions were made and we jumped into the development process.

Beginning the development

You will need to download and install Microsoft Visual Studio 2012 Express for Windows 8 in order to be able to follow the steps that I am describing in this series. The development has to be done on Windows 8, because the end result is a Windows Store application that relies on Windows 8 APIs. Though specific hardware is not required, both ARM and x86 devices will work well, so whether you have a Microsoft Surface RT or Samsung Series 7 slate, you will be able to test the code when you have the opportunity.

To get started, open Visual Studio 2012 and select the C++ project types. You will notice that there are several options you can choose from. You want to create a Windows Store application, so choose the appropriate category. Windows Store applications run in a sandbox, outside the boundaries of the standard .NET runtime.

Next, select the Direct2D (XAML) app type from the project list. This is a new application type introduced in Visual Studio 2012 that allows developers to combine native DirectX graphics with XAML overlays. Do not be confused by the fact that there is a Direct2D in the name—you can still invoke DirectX capabilities supported in the WinRT sandbox:

At this point you might be wondering, why choose a hybrid application instead of a fully-native project? The reason behind this decision is that is allows the developer to focus more on fine-tuning the gameplay instead of creating the UI core in pure DirectX. Because XAML is a part of the game, we can create dynamic UI layouts for the HUD, settings charm and menus without touching the graphical backbone. It is possible to do the same directly through DirectX and was a perfect approach for FallFury, considering the time constraints and the fact that I needed minimal UI overlays. Most of the graphics were already processed through the DirectX pipeline, so I did not have to invest significant resources and time into designing low-level structures for the interactivity layer.

FallFury uses XAML for the following:

• Menus – depending on the screen, the user is able to trigger a number of actions. For example, when the game starts, the user might select the New Game option or decide to take a look at the About screen. In Paused mode, the menu is used to resume the game, adjust settings, or possibly skip a level.
• Game HUD (score indicator, pick-up indicator, health indicator) – during the gameplay, the user is interested in keeping track of where the character is and what is the state of it. The game HUD is shown in active game mode.
• Settings charm extensions – the way the Settings charm works, the OS provides the core harness to hook to the Settings popup. Once shown, it is up to the developer to provide a multitude of options that customize the application behavior; any additional popups shown on selection should be designed individually in XAML.
• User notification – when something happens that can potentially affect the gameplay, the user should be alerted. The core framework provides the capabilities to use a MessageDialog, but in some cases it might not be enough. For example, if new levels are available for download, the user might want to check those out in a custom popup including full previews rather than just text.

When you create the project, a default infrastructure that prints text on the screen—both through Direct2D and XAML—is available. Direct2D is a subset of DirectX APIs and facilitates hardware-accelerated 2D graphics processing. It is used to create basic geometry elements and text. Since I am here working mostly with XAML, I am not going to cover Direct2D in-depth in this article:

I will go into more details regarding the XAML and DirectX interaction model later in the series, but for now, take a look at which parts of the project you have available. First and foremost, you probably notice the combination of both C++ source and header files and DirectXPage.xaml. Starting with Visual Studio 2012, you are now able to create XAML applications in C++. So even if you are an experienced C++ developer who never worked with the Extensible Application Markup Language, you can create the product core in your familiar environment and either import existing XAML structures or delegate the XAML writing to a designer.

If you open the XAML page, you will notice one significant change that you haven’t experienced in standard XAML applications, such as WPF or Silverlight for Windows Phone:

<SwapChainBackgroundPanel x:Name="SwapChainPanel" PointerMoved="OnPointerMoved" PointerReleased="OnPointerReleased">
    <TextBlock x:Name="SimpleTextBlock" HorizontalAlignment="Center" FontSize="42" Height="72" Text="Hello, XAML!" Margin="0,0,0,50"/>
</SwapChainBackgroundPanel>

A SwapChainBackgroundPanel is a component that lets the developer overlay XAML on top of the core DirectX-based experience. Look, for example, at the FallFury main menu as the game runs in landscape mode:

The menu buttons, the label and the side curtains are designed and rendered entirely in XAML. The clouds in the background, as well as the teddy bear, are rendered directly through the DirectX stack. The end-user does not notice any difference in the way these elements interact or are displayed. From a development perspective, however, there are several conditions that must be met.

There can only be one instance of SwapChainBackgroundPanel per app. Therefore, you can have only one overlaid XAML controls set. This does not mean that you can’t have multiple controls, but it implies that to do so you have to implement a control management flow that handles content adaptation. For example, if I invoke the pause state in the game, I don’t show the HUD but rather the screen-specific controls that let me resume or abandon the game and the PAUSE label. This switch needs to be handled on both the DirectX and XAML because a state change affects what is shown on the screen and what behaviors are tracked. As you will see through this series, this is not too hard to implement with a helper class that will store the global game state, that can be accessed from anywhere in the game.

When using SwapChainBackgroundPanel, remember that XAML is in all cases overlaid on top of the DirectX renders. So, no matter what controls you are using, those will always be placed on top of what DirectX shows to the user. For more details about how DirectX and XAML interoperate, check out this MSDN article.

Assets for the game

As with any other game, there is not only code involved in production—there are also sound and graphical assets that create a unique experience for the user. FallFury includes a wide variety of graphical assets designed by Toledo Design as well as audio created by David Walliman.

It is important that all graphical resource requirements are established at the very beginning of development. As I mentioned in the Design section of this article, I had to put together a list of all the game screens, power-ups, obstacles, backgrounds, character states and possible particles that were generated from a texture. That way, when the designer started creating the assets, all components blended together well and their styles were compatible with the vision of the game.

While working on the assets, Arturo Toledo created multiple variations of the same set up that showed how assets integrate in different game conditions:

As you follow this series, you will not have to create your own game assets—we at Coding4Fun decided to provide all graphical and audio assets, which you can download at http://fallfury.codeplex.com/.  We not only provided you with the final PNG and DDS files, but also with the raw assets that can be used in Microsoft Expression Design and Adobe Illustrator. Let’s take a look at what is in the package.

You will notice that the project Assets folder is split in several subfolders. All these assets are used mostly in the XAML layer or in game conditions where texture/sound processing is not necessary. There is also an additional asset folder I will discuss later.

• Backgrounds –the level backgrounds, such as the blue, purple or red sky, as well as the overlays, such as clouds that move simultaneously with the backgrounds. Each background/overlay combination is assigned to a specific level type.
• Bear – some of the bear elements that are displayed at different times in the game, such as when the game is over or when the player wins the entire level set.
• HUD – basic elements that are displayed during the game.
• Icons – the application icons, in a variety of sizes, required for a Windows Store application.
• Medals – winning players are awarded a medal. It can be golden, silver, or bronze.
• Misc – you get some branding elements as well as some graphics that are used in combination with other game items, such as particles.
• Monsters – monsters are used in the “How To” part of the game.
• Music – the long tracks used in different levels and screens.
• Sounds – short sound clips used when different game behaviors in the game are triggered. For example, when the bear gets hit, he lets out a brief cry.

All graphical assets mentioned here are PNG resources. The way I structured the game, some elements are larger than the others and I needed to take some measures to cut down on the package size. PNGs are fairly well-sized without much quality loss compared to raw images. At the beginning of the development process, all graphics were stored in DDS files.

DirectDraw Surface, or DDS, is a file format developed by Microsoft that helps developers optimize some of the graphics by avoiding re-compression performance loss. One of the benefits of using DDS files is the fact that whenever they are processed by the GPU, the amount of memory taken by them is the same as the file size of the DDS file itself. Usually, for 2D games the compression-based performance loss is not necessarily noticeable. For PNG files used on the DirectX stack, resources are allocated to decompress the texture. Not so for DDS files.

Depending on the case, DDS files can be generated on the fly. FallFury preserves relatively small textures in DDS format and larger ones, such as backgrounds, in PNG. It’s the best of both worlds. DDS files can be generated by a tool bundled with the DirectX SDK called dxtex (usually located in C:\Program Files (x86)\Microsoft DirectX SDK (June 2010)\Utilities\bin\x64):

There is also texconv that allows you to create DDS components from the command line, but we’ll explore that later in the series.

All DDS assets are located in the DDS folder in the solution. All that’s there are assets related to levels, such as obstacles, and assets related to character behavior, such as bear states and used weapons and powerups.

Unlike graphic assets that can be mocked from the very start, audio assets are a bit harder to come up with, mainly because at that point you need to be sure what the game will be like at the end. Music and effects should go together with the overall game feel. There are two types of audio components in FallFury—the music and the action-based sound effects. Music is played constantly, whether in the game or on a game screen such as the main menu, unless disabled by the user.

The way the audio engine works in FallFury, files are handled differently depending on where in the game they are used. All music is stored in MP3 files and the short sound effects are stored in uncompressed WAV files.

As a part of the new project that you are creating, replicate the folder structure for the Assets and DDS folders and add them to the solution. The way C++ project references work, you might want to switch to the “Show All Files” mode in the Solution Explorer:

Then, simply copy the folders to the solution folder itself. That way you will have all these resources as a part of the project itself and not just links to external files.

Source Control

It’s a really good idea to use source control. There are multiple options available at no cost, such as CodePlex, Assembla, and Team Foundation Service. You need source control for multiple reasons. The most important reason of them all, however, is that code will break. There were multiple situations where I changed parts of the project and all of a sudden some components stopped working. With the project hooked to a source control system, all I needed was to do a quick rollback to the previous check-in and I was good to go.

A good practice I learned from Clint Rutkas is performing atomic check-ins. When something goes wrong, it is much easier to go back to the check-in where only 20 or 30 code lines were modified from what is currently in the stack, compared to going back to the solution where you will be missing two or more entire source files.

Modular Design & Prototyping

As you are following this project creation from scratch, notice how the entire code base is modular and interchangeable. If I decide to create a new game screen, I can do so easily by inheriting from an existing base class that provides the basic harness. If I want to replace a character model, I can do so by modifying a single class without breaking the entire interaction model.

Prototyping is also a big part of FallFury and it is key that no time is wasted working on features that will have to be entirely replaced or re-written. A good example to this scenario happened just as I started writing the project code. I noticed that the Settings charm required some XAML work for secondary popups that extended from the OS-invoked layer. As I was working on a pure DirectX application, I had to create the XAML in code-behind and that was one of the experiences that could’ve been avoided in a hybrid application. During the prototyping stage it is easy to spot potential integration problems and later change parts of the project to work better together. It is much more complicated to replace project components when the core is wired-in than when you have small parts that independently show how a part of the game works.

Hardware

As you are about to start writing large amounts of code, you are probably wondering whether actual hardware is needed to test the application. The way FallFury was designed, you will be able to run it on any Windows 8 compatible machine, whether a desktop computer or a tablet. If the game is being run on the desktop, I assume that you probably do not have access to an accelerometer or a touch display. If you are running the game on the tablet, you probably do not have a physical keyboard constantly attached to it. These facts ultimately affect how you experience the game itself. From a development perspective, it is important to have multiple types of target hardware available, because the game might behave differently depending on the device configuration. But it is not required to follow this article series. As long as you have a Windows 8 machine, you are good to go.

Conclusion

Now onto 11 more articles on how to build Fall Fury!

For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles!

Prior to DirectX 11.1, stereoscopic 3D required specific hardware and a custom API written for that hardware.  With DX11.1, stereoscopic 3D has been "democratized."  Any GPU that supports DirectX 11.1 can be connected to any device which supports stereoscopic 3D, be it a projector, an LCD TV, or anything else.  Plug it in and DirectX does the rest.

From the software side of things, any DX11.1 application can determine if the connected display supports stereoscopic 3D, and choose to render itself separately for the player's left and right eye.

To showcase this feature, we decided to build a very simple game that would give the illusion of depth, but be easy to explain and play.  What's easier than Pong?  So, we built the world's most over-engineered game of 3D Pong named Maelstrom.

Software

Each player setup consists of two applications: the DirectX 11.1 game written in C++, and the touch-screen controller written in C#/XAML.  Both are Windows Store applications.  Since this is a two player game, there are two instances of each application running, one per player.  All for applications are networked together using StreamSockets from the Windows Runtime.  The two controllers and player two's DirectX game connect to player one's DirectX game, which acts as the "master".  Controller input is sent here, and, once the ball and paddle positions are calculated, the data is drawn for player one and sent to player two which draws the world from the other player's perspective.

Direct3D Application

Getting Started with stereoscopic DirectX11, C++, and XAML

If you have never worked with DirectX before, it can be a little overwhelming at first. And even if you have worked with it some in the past, targeting the new Windows 8 ecosystem, along with C++ and XAML have added some additional changes in how you may have designed your solution previously.

Fortunately, the Windows Dev Center for Windows Store Apps has some great samples to get you started, and we took full advantage of them to get to speed. For a great, simple example of how to leverage the new stereoscopic feature in Direct3D 11.1, we started with Direct3D Stereoscopic Sample which shows the basic adjustments to the Render loop for toggling your virtual cameras. However, to see a great example of a simple game structure that also leverages stereoscopic rendering where available, the tutorial found at Walkthrough: a simple Windows Store game with DirectX is invaluable. Further in this article, we will dive deeper into the specifics of stereoscopic rendering in our game.

One thing to note, if you follow the link in the above Walkthrough to the original project, it will take you to a C++ only implementation of the game. Now, of course, all the DirectX game objects such as the paddle, puck and walls are all rendered using D3D. However, for HUD (Heads up Display) elements, this C++ only sample also uses DirectX exclusively. If you are coming from a managed code background, this will definitely seem like unnecessary overhead. That is because this C++ only sample was created after last year's BUILD conference in 2011 and C++ and DirectX still did not play well with XAML.

However, a few months later, the ability to nest DirectX content in a XAML project became available for true hybrid style solutions (see the article DirectX and XAML interop - Windows Store apps using C++ and DirectX for more information). After this feature was added, the simple Shooter Game referenced above had its HUD logic rewritten in XAML and posted up to Dev Center as XAML DirectX 3D shooting game sample, which shows both stereoscopic support, a simple Game Engine structure in C++ and XAML integration. At this point, we had all the starter code we needed to start writing our own game.

Game Engine

We modified the base sample to accommodate our needs.  We created specific GameObjects, such as Paddle, Puck, etc. to add the behaviors we needed.  We also added an Update and Render method to the base GameObject so that, for every frame, we could do any calculations required, and then draw the object to the screen.  This is very similar to how XNA sets up its game engine.

Game Constants

Because we were tweaking a variety of values like colors, sizes, camera locations, etc., we created a GameConstants.h header file which contains nothing but these types of values in a single location.  This made it very easy for us to quickly try out various tweaks and see the results on the next run.  Using namespaces helped keep the code a bit more manageable here as well.  Here’s a quick snippet of that file:

namespace GameConstants
{
    // bounds of the arena
    static const DirectX::XMFLOAT3 MinBound    = DirectX::XMFLOAT3( 0.0f,  0.0f,  0.0f);
    static const DirectX::XMFLOAT3 MaxBound    = DirectX::XMFLOAT3(19.0f, 10.0f, 90.0f);

    // game camera "look at" points
    static const DirectX::XMFLOAT3 LookAtP1    = DirectX::XMFLOAT3(9.5f, 5.0f, 90.0f);
    static const DirectX::XMFLOAT3 LookAtP2    = DirectX::XMFLOAT3(9.5f, 5.0f,  0.0f);

    // Waiting Room camera positions
    static const DirectX::XMFLOAT3 WaitingEyeP1                = DirectX::XMFLOAT3(GameConstants::MaxBound.x/2, GameConstants::MaxBound.y/2, GameConstants::MaxBound.z - 12.0f);
    static const DirectX::XMFLOAT3 WaitingEyeP2                = DirectX::XMFLOAT3(GameConstants::MaxBound.x/2, GameConstants::MaxBound.y/2, GameConstants::MinBound.z + 12.0f);
    static const DirectX::XMFLOAT3 WaitingEyeMjpegStation    = DirectX::XMFLOAT3(GameConstants::MaxBound.x/2, GameConstants::MaxBound.y/2, GameConstants::MinBound.z + 9.6f);

    // game camera eye position
    static const DirectX::XMFLOAT3 EyeP1        = DirectX::XMFLOAT3(GameConstants::MaxBound.x/2, GameConstants::MaxBound.y/2, GameConstants::MinBound.z - 6.0f);
    static const DirectX::XMFLOAT3 EyeP2        = DirectX::XMFLOAT3(GameConstants::MaxBound.x/2, GameConstants::MaxBound.y/2, GameConstants::MaxBound.z + 6.0f);

    static const float Paddle2Position    = MaxBound.z - 5.0f;

    namespace PaddlePower
    {
        // power level to light paddle at maximum color
        static const float Max                    = 9.0f;

        // max paddle power color...each component will be multiplied by power factor
        static const DirectX::XMFLOAT4 Color    = DirectX::XMFLOAT4(0.2f, 0.4f, 0.7f, 0.5f);
    
        // factor to multiply mesh percentage based on power
        static const float MeshPercent            = 1.2f;
    };
    
    // time to cycle powerups
    namespace Powerup
    {
        namespace Split
        {
            static const float Time                        = 10.0f;
            static const float NumTiles                    = 4;
            static const DirectX::XMFLOAT4 TileColor    = DirectX::XMFLOAT4(0.1f, 0.4f, 1.0f, 1.0f);
            static const float TileFadeUp                = 0.20f;
            static const float TileDuration                = 2.10f;
            static const float TileFadeDown                = 0.20f;
            static const float TileMeshPercent            = 2.0f;
            static const float TileDiffusePercent        = 2.0f;
        };
    };
}

Stereoscopic 3D

Direct3D must be initialized properly to support stereoscopic displays.  When the swap chains are created, an additional render target is required, such that one render target is for the left eye, and one render target is for the right eye.  Direct3D will let you know if a stereoscopic display is available, so you can create the swap chain and render targets appropriately.

With those in place, it’s simply a matter of rendering your scene twice, once per eye…that is, once per render target.

For our game this was very simple.  Our in-game camera contains two projection matrices, one representing the view from the left eye, and one from the right eye.  These are calculated when the projection parameters are set.

void Camera::SetProjParams(
    _In_ float fieldOfView,
    _In_ float aspectRatio,
    _In_ float nearPlane,
    _In_ float farPlane
    )
{
    // Set attributes for the projection matrix.
    m_fieldOfView = fieldOfView;
    m_aspectRatio = aspectRatio;
    m_nearPlane = nearPlane;
    m_farPlane = farPlane;
    XMStoreFloat4x4(
        &m_projectionMatrix,
        XMMatrixPerspectiveFovLH(
            m_fieldOfView,
            m_aspectRatio,
            m_nearPlane,
            m_farPlane
            )
        );

    STEREO_PARAMETERS* stereoParams = nullptr;
    // Update the projection matrix.
    XMStoreFloat4x4(
        &m_projectionMatrixLeft,
        MatrixStereoProjectionFovLH(
            stereoParams,
            STEREO_CHANNEL::LEFT,
            m_fieldOfView,
            m_aspectRatio,
            m_nearPlane,
            m_farPlane,
            STEREO_MODE::NORMAL
            )
        );

    XMStoreFloat4x4(
        &m_projectionMatrixRight,
        MatrixStereoProjectionFovLH(
            stereoParams,
            STEREO_CHANNEL::RIGHT,
            m_fieldOfView,
            m_aspectRatio,
            m_nearPlane,
            m_farPlane,
            STEREO_MODE::NORMAL
            )
        );
}

Depending on which eye we are rendering, we grab the appropriate projection matrix and pass it down to the vertex shader, so the final scene is rendered offset for the proper eye.

Collision Detection

If you are just starting to move into 3D modeling and programming, one of the trickier aspects of your game can be collision detection and response. Maelstrom uses primitives for all of the game elements, so our collision code was able to be a bit more straightforward compared to complex mesh collisions, but understanding a few core math concepts is still critical to grasp what the code is doing.

Fortunately, DirectX provides us with an DirectX Math Library that is able to do the serious heavy lifting, so the main complexity comes from framing the problem and learning how to apply the library.

For example, In our situation we had up to three very fast moving spheres and needed to check for wall collisions and then handle to appropriate bounce, since some of the walls would also be angled. In a 2D game, a collision detection between a sphere and an axis line is very easy. If the distance between a circle and the line is less than or equal to the radius of the sphere, they are touching. On every frame, you move your circle based on its velocity and do your collision test again. But even here, your solution may not be that easy for two reasons.

First, what if the line is angled and not lying flat on the X or Y axis? You have to find the point on the line based on the line's angle that is closest to the sphere to do your distance calculations. And if you then want it to bounce, you have to rotate the velocity of the circle by the line's angle, calculate your bounce, and then rotate back. And that's just rotated walls in 2D. When you move up to 3D, you have to take into account the surface normal  (which way the 3D plane is facing) in your calculations.

The second complexity that we needed to account for and which pops up in either 2D or 3D collision detection is travel between frames. In other words, if your ball is travelling very fast, it may have completely travelled through your collision boundary in between frames and you wouldn't notice it if you are only doing a distance / overlap check as outlined above. In our case, the pucks had the ability of travelling very fast with a speed boost, so we needed a more robust solution. Therefore, instead of implementing a simple sphere plane intersection test, we needed to create a line of motion that represented where the ball ended on the previous frame and where it currently is after it's new velocity is added to it's position. That line then needs to first be tested to see if it crosses a WallTile. If it does cross, then we know an collision has occurred between frames. We then solve for the time (t) between frames the Sphere would have first made contact to know the exact point of impact and calculate the appropriate "bounce off" direction.

The final code for a puck (or moving sphere) and wallTile collision test looks like this:

bool GameEngine::CheckWallCollision(Puck^ puck)
{
    bool isIntersect = false;
    bool wallCollision = false;

    for(unsigned int i = 0; i < m_environmentCollisionWalls.size(); i++)
    {
        WallTile^ wall = m_environmentCollisionWalls[i];
        float radius = puck->Radius();
        float signedRadius = puck->Radius();
        float contactTime = 0.0f;
        XMVECTOR contactPlanePoint = XMVectorSet(0.0f, 0.0f, 0.0f, 0.0f);
        XMVECTOR contactPuckPosition = XMVectorSet(0.0f, 0.0f, 0.0f, 0.0f);

        bool intersectsPlane = false;

        // Determine the velocity of this tick by subtracting the previous position from the proposed current position.
        // in the previous update() cycle, puck->Position() = puck->OldPosition() + ( puck->velocity * timerDelta ).
        // Therefore, this calculated velocity for the current frame movement differs from the stored velocity
        // since the stored velocity is independent of each game tick's timerDelta.
        XMVECTOR puckVectorVelocity = puck->VectorPosition() - puck->OldVectorPosition();
        float D = XMVectorGetX( XMVector3Dot( wall->VectorNormal(), wall->VectorPosition() ) );

        // Determine the distance of the puck to the plane of the wall.
        float dist = XMVectorGetX( XMVector3Dot(wall->VectorNormal(), puck->OldVectorPosition() )) - D;
        signedRadius = dist > 0 ? radius : -radius;

        // if the distance of the puck to the plane is already less than the radius, the oldPosition() was intersecting already
        if ( fabs(dist) < radius )
        {
            // The sphere is touching the plane.
            intersectsPlane = true;
            contactTime = 0.0f;
            contactPuckPosition = puck->OldVectorPosition();
            contactPlanePoint = puck->OldVectorPosition() + wall->VectorNormal()*XMVectorSet(signedRadius,signedRadius,signedRadius,1.0f);
        }
        else
        {
            // See if the time it would take to cross the plane from the oldPosition() with the current velocity falls within this game tick.
            // puckVelocityNormal is the amount of force from the velocity exerted directly toward the plane.
            float puckVelocityNormal = XMVectorGetX(XMVector3Dot(wall->VectorNormal(), puckVectorVelocity ));
            // if the puckvVelocityNormal times the distance is less than zero, a plane intersection will occur
            if ( puckVelocityNormal * dist < 0.0f )
            {
                // determine the contactTime, taking into account the shell of the sphere ( position() + radius )
                // is what will make contact, not the position alone.
                contactTime =  (signedRadius - dist) / puckVelocityNormal;
                // if the contact time is bewteen zero and one, the intersection has occured bewteen oldPosition() and position()
                if ( contactTime > 0.0f && contactTime < 1.0f )
                {
                    intersectsPlane = true;
                    // this is the position of the puck when its shell makes contact on the plane
                    contactPuckPosition = puck->OldVectorPosition() + XMVectorScale(puckVectorVelocity, contactTime);
                    // this is the position on the plane where the shell touches.
                    contactPlanePoint = contactPuckPosition - XMVectorScale(wall->VectorNormal(), signedRadius);
                }
            }
        }

        // If the puck has contacted the wall plane, determine if the point of contact falls within the wall boundary for true contact.
        if (intersectsPlane)
        {
            float Kr = 1.0f; // Kr is the coefficient of restitution. At 1.0, we have a totally elastic bounce with no dampening. At Kr = 0.0, the ball would stop at the wall.

            // Make sure the puck velocity and wall normal are facing each other
            float impact = XMVectorGetX ( XMVector3Dot ( wall->VectorNormal(), puck->VectorVelocity()) );
            if (impact < 0.0f)
            {
                wallCollision = true;

                //// bounce the vector off the plane
                XMVECTOR VectorNormal = XMVector3Dot(wall->VectorNormal(), puck->VectorVelocity())*wall->VectorNormal();
                XMVECTOR VectorTangent = puck->VectorVelocity() - VectorNormal;

                puck->Velocity(VectorTangent - (XMVectorScale(VectorNormal, Kr)));
                puck->Position(contactPuckPosition);

                int segment = (int)(puck->Position().z / GameConstants::WallSegmentDepth);
                segment = max(min(segment, GameConstants::NumWallSegments-1), 0);

                auto tiles = m_wallTiles[segment];
                WallTile^ tile = tiles[i];
                if(tile->GetPowerup() == Powerup::Split)
                    SplitPucks();

                break;
            }
        }
    }

    return wallCollision;
}

Drawing Maelstrom

To draw the game, we wanted to use some advanced techniques. We decided to go with a light pre-pass deferred rendering pipeline with normal mapping. That’s a lot of jargon but it isn’t all that complicated once you know what the jargon means, so let’s break it down.

When you draw something in 3D, there are three things that come together to determine the final color of each pixel on the screen: meshes, materials, and lights. A mesh is a collection of triangles that make up a game object (such as a wall tile in Maelstrom). On its own, a mesh is just a bunch of dots and lines. A material makes a mesh look like something. It could be as simple as a solid color but usually it’s a texture and sometimes it’s more (the wall tiles in Maelstrom use both a texture and a normal map to define their material properties). Lastly, lights transform materials by determining how bright they should appear and what sort of tint, if any, they should have. Without lights you would either have complete darkness or you would have flat lighting (where everything has a uniform brightness and adding a tint color would uniformly tint everything on the screen).

Forward Rendering vs. Deferred Rendering vs. Light Pre-Pass Rendering

The simplest approach to drawing 3D graphics is something called forward rendering. With forward rendering, drawing consists of rendering the mesh and calculating its material and all the lights that affect the material all at the same time. The more lights you add, the more complicated your shaders become since you have to determine whether each light affects the material and if so how much. (Ok, so there’s also multi-pass forward rendering, but that has its own problems – more passes mean longer render times and thus a lower frame rate – and we wanted to keep the descriptions simple).

In the last 5 years, many games started using a technique called deferred rendering. In classic deferred rendering, there are two rendering passes. The first pass renders the positions, normals, and material values of all the meshes in the scene to something called a G-Buffer (two or more render targets); nothing is actually drawn to the screen in this first pass. The second pass uses the data from the G-Buffer (which tells us everything we need to know about the geometry that appears at each screen pixel) and combines it with the lights to create the final image that you see. By doing this, we decouple geometry and lighting. This makes it possible to add more lights to the scene with a much smaller performance impact than in forward rendering since we don’t need to create a really complex pixel shader to handle all the lights (single-pass forward rendering) or draw the geometry over and over again for each light (multi-pass forward rendering).

There are drawbacks to classic deferred rendering though. Even a minimal G-Buffer takes up quite a bit of memory and the more different types of materials you want to support, the larger the G-Buffer will need to be. Wolfgang Engel, an XNA/DirectX MVP, came up with a variation on deferred rendering which he called Light Pre-Pass Rendering. This is a three pass technique. We once again use a G-Buffer, but in this case it is smaller than the classic deferred rendering G-Buffer and can even be squeezed down to a single render target which makes it viable for graphics hardware which does not support drawing to multiple render targets at the same time.

The G-Buffer is created in the first pass by rendering all the scene geometry. It only needs to store normals and the geometry’s world position. We stored the world position of the geometry at that screen position in one render target and its normal at that screen position in second render target for simplicity.

The next pass draws the lights to a light accumulation buffer. The buffer starts out entirely dark and each light that is rendered adds brightness (and tint, if any) to the light buffer. These lighting calculations take into account the normal and world position of the geometry that is at each screen position, drawing the values from the G-Buffer, such that each light only affects the pixels it is supposed to have an impact on. In Maelstrom we ended up only using point lights (spheres of light that fade out as you get further from the light’s position), but you can use any kind of light you can imagine (spot lights and directional lights are the two other common light types). Adding more lights has a very low impact on rendering time and this kind of lighting tends to be much easier for the designer to work with since there’s no need for him or her to understand HLSL or even any complicated C++ in order to add, remove, reposition, or otherwise change any lights.

The final pass draws the geometry a second time. This time, though, all the lighting calculations are done so all we do here is just render the meshes with their appropriate materials, adjust the color values and intensities from the material based on the light buffer value, and we’re done. Each rendering style (forward, deferred, and light pre-pass) has its own benefits and drawbacks, but in this case light pre-pass was a good solution and choosing it let us show how a state-of-the-art graphics technique works.

Normal Mapping

We also incorporated normal mapping. Normal mapping makes us of a special texture (a normal map) in addition to the regular texture that a material has. Normals are values used in lighting calculations to determine how much a particular light should affect a particular pixel. If you wanted to draw a brick wall, you would typically create two triangles that lined up to form a rectangle and you would apply a texture of a brick wall to them as their material. The end result of that doesn’t look very convincing though since unlike a real brick wall there are no grooves in the mortared area between each brick since our brick and mortar is just a flat texture applied to flat triangles. We could fix this by changing from two triangles to a fully modeled mesh with actual grooves, but that would add thousands of extra vertices which would lower the frame rate.

So instead we use a normal map, which fakes it. One of the reasons that the two triangles + a brick wall texture approach doesn’t look right is because the lighting doesn’t behave correctly when compared to a real brick wall (or to a fully modeled mesh of a real brick wall). The normals point straight out perpendicular from the face of the rectangle whereas if we had the fully modeled mesh with actual grooves, the surface normals would only point straight out on the bricks themselves and they would curve along the mortared areas such that the lighting calculations would end up giving us the right levels of light and dark depending on the location and direction of the light. That’s where a normal map comes in. The normal map (which you can generate using a plugin for Adobe Photoshop or GIMP or by modeling a real brick wall in 3DSMax, Maya, or Blender which you then “bake” a normal map from) allows us to get the same lighting effect as we would with a fully modeled mesh while still keeping the simple two triangle + a brick wall texture approach that gives us really good performance for our game. There are limits to the effectiveness of normal mapping (you can’t use it to fake anything too deep and it doesn’t hold up as well if the camera can get really close to the object) but in Maelstrom it allowed us to keep the walls as simple triangles (like the two triangles + a brick wall texture example above) while making it seem like there were geometric grooves in the wall. Here’s a before and after screenshot using normal mapping:

Post-Processing Effects

We also used several post-processing effects. The first was the bloom effect. Bloom is an effect that analyzes a rendered image, identifies parts that are above a certain brightness threshold, and makes those areas brighter and adds a peripheral glow to them as well, giving it a look and feel that is similar to a neon sign or to the light cycles in the movie Tron. Here’s the same shot as above with the addition of bloom:

We also made use of two different damage effects. Whenever the player took damage, we had a reddish tinge around the edge of the screen. This was simply a full screen overlay texture that is actually white but is tinted red by the shader. It is alpha-blended over the final rendered scene and fades out over the course of a couple of seconds. Rather than fading out linearly, we use a power curve which helps to sell the effect as being more complicated than it really is.

Lastly we added in some damage particles. The particles themselves were created using a geometry shader. The vertex shader took in a series of points in world space and passed these points along to the geometry shader. The geometry shader expanded these points into two triangles by generating the missing vertices and applying the world-view-projection transformation matrix to transform the positions from world coordinates to homogeneous coordinates so that they can then be rasterized correctly by D3D and the resulting pixels passed along to the pixel shader. Once again we used a simple texture with alpha blending to simulate much more complicated geometry than we were actually drawing. In this case we also made use of a texture atlas (an image made up of smaller images) which, in conjunction with the randomizer we used to generate the initial vertices for the particles, allowed us to have several different particle textures. Like with the power curve for the damage texture, the texture atlas allowed us to make the particles seem more complex than they really were. It also let us show off the use of a geometry shader, a feature that was added in DirectX 10 and requires DirectX 10 or higher hardware.

Audio

All audio was done using the XAudio2 API.  Thankfully, we were able to get a huge head start by using some of the code from the sample project we started from.  The audio engine sets up the very basics of XAudio2, and then wraps that with a simpler API for the rest of the application to call.

We don’t have many sound effects, so we on startup, we load all sounds effects and music cues into a std::map, keyed on a SoundCue enum.  Sounds are loaded using the Media Foundation classes, and the resulting byte data of the sound (and some format information) are stored in our SoundEffect class.

void AudioEngine::Initialize()
{
    m_audio = ref new Audio();
    m_audio->CreateDeviceIndependentResources();

    m_mediaReader = ref new MediaReader();

    // Impacts
    m_soundMap[SoundCue::BallLaunch]    = LoadSound("Sounds\\Impacts\\BallLaunch.wav");
    m_soundMap[SoundCue::Buzz]          = LoadSound("Sounds\\Impacts\\Buzz.wav");
    m_soundMap[SoundCue::Impact1]       = LoadSound("Sounds\\Impacts\\Impact1.wav");
    m_soundMap[SoundCue::Impact2]       = LoadSound("Sounds\\Impacts\\Impact2.wav");

...

}

SoundEffect^ AudioEngine::LoadSound(String^ filename)
{
    Array<byte>^ soundData = m_mediaReader->LoadMedia(filename);
    auto soundEffect = ref new SoundEffect();
    soundEffect->Initialize(m_audio->SoundEffectEngine(), m_mediaReader->GetOutputWaveFormatEx(), soundData);
    return soundEffect;
}

When the game needs to play a sound, it simply calls the PlaySound method, passing in the cue to play, and the volume to play it at.  PlaySound keys into the sound map, getting the associated SoundEffect, and plays it.

void AudioEngine::PlaySound(SoundCue cue, float volume, bool loop)
{
    m_soundMap[cue]->Play(volume, loop);
}

MJPEG Cameras

To achieve the effect of seeing the opponent in stereoscopic 3D, we strapped two Axis M1014 network cameras side-by-side.  Using Brian’s MJPEG Decoder library, with a special port to Windows Runtime, individual JPEG frames were pulled off each camera, and then applied to a texture at the back of the arena.  The image from the left camera is drawn when DirectX renders the player’s left eye, and the frame from the right camera is drawn when DirectX renders the right eye.  This is a cheap and simple way to pull off live stereoscopic 3D.

void MjpegCamera::Update(GameEngine^ engine)
{
    if(m_decoderLeft != nullptr)
        UpdateTexture(m_decoderLeft->CurrentFrame, &textureLeft);

    if(m_decoderRight != nullptr)
        UpdateTexture(m_decoderRight->CurrentFrame, &textureRight);

    Face::Update(engine);
}

void MjpegCamera::Render(_In_ ID3D11DeviceContext *context, _In_ ID3D11Buffer *primitiveConstantBuffer, _In_ bool isFirstPass, int eye)
{
    if(eye == 1 && textureRight != nullptr)
        m_material->SetTexture(textureRight.Get());
    else if(textureLeft != nullptr)
        m_material->SetTexture(textureLeft.Get());

    GameObject::Render(context, primitiveConstantBuffer, isFirstPass);
}

With the distance between the cameras being about the distance of human eyes (called the intra-axial distance), the effect works pretty well!

Tablet/Controller

The Tablet controller is the touch screen that lets the player control their 3D paddle in the Game Console app. For this part of the game system, there wasn't a reason to dive deep into DirectX and C++ since the controller is neither stereoscopic or visually intense, so we kept things simple with C#.

Since the controller would also serve as our attract screen in the podium to entice potential players, we wanted to have the wait screen do something eye-catching. However, if you are moving from C# in WPF to C# and XAML in WinRT and are used to taking advantage of some of the more common "memory hoggish UX hacks" from WPF, you'll quickly find them absent in WinRT! For example, we no longer have OpacityMask, non-rectangular clipping paths or the ability to render a UIElement to a Bitmap. Our bag of UX tricks may be in need of an overhaul. However, what we do get in C# / XAML for WinRT is Z rotation, which is something we've had in Silverlight but I personally have been begging for in WPF for a long time.

Therefore, the opening animation in the controller is a procedurally generated effect that rotates PNG "blades" in 3D space, creating a very compelling effect. Here is how it works. The Blade user control is a simple canvas that displays one of a few possible blade images. The Canvas has a RenderTransform to control the scale and rotation and a PlaneProjection which allows us to rotate the blade graphic in X, Y and Z space.

<Canvas>
    <Canvas.RenderTransform>
        <ScaleTransform x:Name="scale" ScaleX="1.0" ScaleY="1.0" CenterX="0" CenterY="0"/>
    </Canvas.RenderTransform>
    <Canvas.Projection>
        <PlaneProjection x:Name="projection" CenterOfRotationX="0" CenterOfRotationY="0" CenterOfRotationZ="0" RotationX="00" RotationY="0" RotationZ="0"/>
    </Canvas.Projection>
    <Image x:Name="img0" Width="300" Height="100" Canvas.Left="-4" Canvas.Top="-12" Source="Images/s0.png"/>
</Canvas>

Each Blade is added dynamically to the Controller when the Tablet application first loads, stored in a List to have it's Update() method called during the CompositionTarget.Rendering() loop.

protected override void OnNavigatedTo(NavigationEventArgs e)
{
    canvas_blades.Children.Clear();
    _blades.Clear();
    
    for (int i = 0; i < NumBlades; i++)
    {
        Blade b = new Blade { X = 950.0, Y = 530.0 };
        int id = _rand.Next(0, 5);
        b.SetBlade(id);
        b.Speed = .1 + id * .1;
        SeedBlade(b);
        _blades.Add(b);
        canvas_blades.Children.Add(b);
    }
}

void CompositionTarget_Rendering(object sender, object e)
{
    if(_inGame)
    {
        paddle.Update();
    }
    else if(_isClosing)
    {
        foreach (Blade b in _blades)
            b.UpdateExit();
    }
    else
    {
        foreach (Blade b in _blades)
            b.Update();
    }
}

Since each Blade has been assigned an individual speed and angle of rotation along all three axis, they have a very straightforward Update function. The reason we keep the rotation values between -180 and 180 during the spinning loop is to make it easier to spin them out zero when we need them to eventually leave the screen.

public void Update()
{
   _rotX += Speed;
   _rotZ += Speed;
   _rotY += Speed;

   if (_rotX > 180) _rotX -= 360.0;
   if (_rotX < -180) _rotX += 360.0;
   if (_rotY > 180) _rotY -= 360.0;
   if (_rotY < -180) _rotY += 360.0;
   if (_rotZ > 180) _rotZ -= 360.0;
   if (_rotZ < -180) _rotZ += 360.0;

   projection.RotationX = _rotX;
   projection.RotationY = _rotY;
   projection.RotationZ = _rotZ;
}

public void UpdateExit()
{
   _rotX *= .98;
   _rotZ *= .98;
   _rotY += (90.0 - _rotY) * .1;

   projection.RotationX = _rotX;
   projection.RotationY = _rotY;
   projection.RotationZ = _rotZ;
}

Network

To continue the experiment of blending C# and C++ code, the network communication layer was written in C# as a Windows Runtime component.  Two classes are key to the system: SocketClient and SocketListener.  Player one’s game console starts a SocketListener to listen for incoming connections from each game controller, as well as player two’s game console.  Each of those use a SocketClient object to make the connection.

In either case, once the connection is made, the client and the listener sit and wait for data to be transmitted.  Data must be sent as an object which implements our IGamePacket interface.  This contains two important methods: FromDataReaderAsync and WritePacket.  These methods serialize and deserialze the byte data to/from an IGameState packet of whatever type is specified in the PacketType property.

namespace Coding4Fun.Maelstrom.Communication
{
    public enum PacketType
    {
        UserInputPacket = 0,
        GameStatePacket
    }
    
    public interface IGamePacket
    {
        PacketType Type { get; }
        
        IAsyncAction FromDataReaderAsync(DataReader reader);
        void WritePacket(DataWriter writer);
    }
}

The controllers write UserInputPackets to the game console, consisting of X,Y positions of the paddle, as well as whether the player has tapped the screen to begin.

public sealed class UserInputPacket : IGamePacket
{
    public PacketType Type { get { return PacketType.UserInputPacket; } }
    
    public UserInputCommand Command { get; set; }
    public Point3 Position { get; set; }
}

Player one’s game console writes a GameStatePacket to player' two’s game console, which consists of the positions of each paddle, each ball, the score, and which tiles are lit for the ball splitter power up.  Player two’s Update and Render methods use this data to draw the screen appropriately.

Hardware

The hardware layer of this project is responsible for two big parts.  One is a rumble effect that fires every time the player is hit, and the other is a lighting effect that changes depending on the game state.

As all good programmers do, we reused code from another project.  We leveraged the proven web server from Project Detroit for our Netduino, but with a few changes.  Here, we had static class “modules” which knew how to talk to the physical hardware, and “controllers” which handled items like a player scoring, game state animations, and taking damage.  Because the modules are static classes, we can have them referenced in multiple classes without issue.

NETMF Web Server

When a request comes in, we perform the requested operation, and then return a new line character to verify we got the request. If you don’t return any data, some clients will actually fire a second request which then can cause some odd behaviors.  The flow is as follows:

1. Parse the URL
2. Get the target controller
3. Execute the appropriate action

private static void WebServerRequestReceived(Request request)
{
    var start = DateTime.Now;
    Logger.WriteLine("Start: " + request.Url + " at " + DateTime.Now);
    
    try
    {
        var data = UrlHelper.ParseUrl(request.Url);
        
        var targetController = GetController(data);
        
        if (targetController != null)
        {
            targetController.ExecuteAction(data);
        }
    }
    catch (Exception ex0)
    {
        Logger.WriteLine(ex0.ToString());
    }
    
    request.SendResponse(NewLine);
    
    Logger.WriteLine("End: " + request.Url + " at " + DateTime.Now + " took: " + (DateTime.Now - start).Milliseconds);
}

public static IController GetController(UrlData data)
{
    if (data.IsDamage)
        return Damage;
    
    if (data.IsScore)
        return Score;
    
    if (data.IsGameState)
        return GameState;
    
    // can assume invalid
    return null;
}

Making It Shake

We used a Sparkfun MP3 trigger board, a subwoofer amplifier, and bass rumble plates to create this effect.  The MP3 board requires power, and two jumpers to cause the MP3 to play.  It has an audio jack that then gets plugged into the amplifier which powers the rumble plates.

From here, we just needed to wire a ground to the MP3 player’s ground pin, and the target pin on the MP3 player to a digital IO pin on the Netduino.  In the code, we declare it as an OutputPort and give it an initial state of true.  When we get a request, we toggle the pin on a separate thread.

private static readonly OutputPort StopMusic = new OutputPort(Pins.GPIO_PIN_D0, true);
private static readonly OutputPort Track1 = new OutputPort(Pins.GPIO_PIN_D1, true);
// .. more pins

public static void PlayTrack(int track)
{
    switch (track)
    {
        case 1:
            TogglePin(Track1);
            break;
            // ... more cases
        default:
            // stop all, invalid choice
            TogglePin(StopMusic);
            break;
    }
}

public static void Stop()
{
    TogglePin(StopMusic);
}

private static void TogglePin(OutputPort port)
{
    var t = new Thread(() =>
    {
            port.Write(false);
            Thread.Sleep(50);
            port.Write(true);
    });
    
    t.Start();
}

Lighting Up the Room

For lighting, we used some RGB Lighting strips.  The strips can change a single color and use a PWM signal to do this.  This is different than the lighting we used in Project Detroit which allowed us to individually control each LED and used SPI to communicate.  We purchased an RGB amplifier to allow a PWM signal to power a 12 volt strip.  We purchased ours from US LED Supply and the exact product was RGB Amplifier 4A/Ch for interfacing with a Micro-Controller (PWM/TTL Input).

We alter the Duty Cycle to shift the brightness of the LEDs and do this on a separate thread.  Below is a stripped down version of the lighting hardware class.

public static class RgbStripLighting
{
    private static readonly PWM RedPwm = new PWM(Pins.GPIO_PIN_D5);
    private static readonly PWM GreenPwm = new PWM(Pins.GPIO_PIN_D6);
    private static readonly PWM BluePwm = new PWM(Pins.GPIO_PIN_D9);
    
    private const int ThreadSleep = 50;
    private const int MaxValue = 100;
    
    const int PulsePurpleIncrement = 2;
    const int PulsePurpleThreadSleep = 100;
    
    private static Thread _animationThread;
    private static bool _killThread;
    
    #region game state animations
    public static void PlayGameIdle()
    {
        AbortAnimationThread();
        
        _animationThread = new Thread(PulsePurple);
        _animationThread.Start();
    }
    #endregion
        
            
    private static void PulsePurple()
    {
        while (!_killThread)
        {
            for (var i = 0; i <= 50; i += PulsePurpleIncrement)
            {
                SetPwmRgb(i, 0, i);
            }
            
            for (var i = 50; i >= 0; i -= PulsePurpleIncrement)
            {
                SetPwmRgb(i, 0, i);
            }
            
            Thread.Sleep(PulsePurpleThreadSleep);
        }
    }
    
    private static void AbortAnimationThread()
    {
        _killThread = true;
        
        try
        {
            if(_animationThread != null)
                    _animationThread.Abort();
        }
        catch (Exception ex0)
        {
            Debug.Print(ex0.ToString());
            Debug.Print("Thread still alive: ");
            Debug.Print("Killed Thread");
        }
        
        _killThread = false;
    }
    
    private static void SetPwmRgb(int red, int green, int blue)
    {
        // typically, 0 == off and 100 is on
        // things flipped however for the lighting so building this in.
        
        red = MaxValue - red;
        green = MaxValue - green;
        blue = MaxValue - blue;
        
        red = CheckBound(red, MaxValue);
        green = CheckBound(green, MaxValue);
        blue = CheckBound(blue, MaxValue);
        
        RedPwm.SetDutyCycle((uint) red);
        GreenPwm.SetDutyCycle((uint) green);
        BluePwm.SetDutyCycle((uint) blue);
        
        Thread.Sleep(ThreadSleep);
    }
    
    public static int CheckBound(int value, int max)
    {
        return CheckBound(value, 0, max);
    }
    
    public static int CheckBound(int value, int min, int max)
    {
        if (value <= min)
            value = min;
        else if (value >= max)
            value = max;
        
        return value;
    }
}

Conclusion

We built this experience over the course of around 4 to 5 weeks.  It was our first DirectX application in a very long time, and our first C++ application in a very long time.  However, we were able to pick up the new platform and language changes fairly easily and create a simple, yet fun game in that time period.

Attributions

• Music track - "FatLoad- The Bullet(no master)" by FreaK NeoS
• Sound effects + music edition - David Wallimann
• DirectX shaders - Michael McLaughlin

In early January, we were tasked with creating a unique, interactive experience for the SXSW Interactive launch party with Frog Design. We bounced around many ideas, and finally settled on a project that Rick suggested during our first meeting: boxing robots controlled via Kinect.

The theme of the opening party was Retro Gaming, so we figured creating a life size version of a classic tabletop boxing game mashed up with a "Real Steel"-inspired Kinect experience would be a perfect fit. Most importantly, since this was going to be the first big project of the new Coding4Fun team, we wanted to push ourselves to create an experience that needed each of us to bring our unique blend of hardware, software, and interaction magic to the table under an aggressively tight deadline.

Hardware

The BoxingBots had to be fit a few requirements:

1. They had to be fun
2. They had to survive for 4 hours, the length of the SXSW opening party
3. Each robot had to punch for 90 seconds at a time, the length of a round
4. They had to be life-size
5. They had to be Kinect-drivable
6. They had to be built, shipped, and reassembled for SXSW

Creating a robot that could be beaten up for 4 hours and still work proved to be an interesting problem. After doing some research on different configurations and styles, it was decided we should leverage a prior project to get a jump start to meet the deadline. We repurposed sections of our Kinect drivable lounge chair, Jellybean! This was an advantage because it contained many known items, such as the motors, motor controllers, and chassis material.  Additionally, it was strong and fast, it was modular, and the code to drive it was already written.

Jellybean would only get us part of the way there, however.  We also had to do some retrofitting to get it to work for our new project. The footprint of the base needed to shrink from 32x50 inches to 32x35 inches, while still allowing space to contain all of the original batteries, wheels, motors, motor controllers, switches, voltage adapters. We also had to change how the motors were mounted with this new layout, as well as provide for a way to easily "hot swap" the batteries out during the event. Finally, we had to mount an upper body section that looked somewhat human, complete with a head and punching arms.

 
Experimenting with possible layouts

The upper body had its own challenges, as it had to support a ton of equipment, including:

• Punching arms
• Popping head
• Pneumatic valves
• Air manifold
• Air Tank(s)
• Laptop
• Phidget interface board
• Phidget relay boards
• Phidget LED board
• Xbox wireless controller PC transmitter / receiver
• Chest plate
• LEDs
• Sensors to detect a punch

Brian and Rick put together one of the upper frames

Punching and Air Tanks

We had to solve the problem of getting each robot to punch hard enough to register a hit on the opponent bot while not breaking the opponent bot (or itself). Bots also had to withstand a bit of side load in case the arms got tangled or took a side blow. Pneumatic actuators provided us with a lot of flexibility over hydraulics or an electrical solution since they are fast, come in tons of variations, won't break when met with resistance, and can fine tuned with a few onsite adjustments.

To provide power to the actuators, the robots had two 2.5 gallon tanks pressurized to 150psi, with the actuators punching at ~70psi.  We could punch for about five 90-second rounds before needing to re-pressurize the tanks.  Pressurizing the onboard tanks was taken care of by a pair of off-the-shelf DeWalt air compressors.

The Head

It wouldn’t be a polished game if the head didn’t pop up on the losing bot, so we added another pneumatic actuator to raise and lower the head, and some extra red and blue LEDs. This pneumatic is housed in the chest of the robot and is triggered only when the game has ended.

To create the head, we first prototyped a concept with cardboard and duct tape. A rotated welding mask just happened to provide the shape we were going for on the crown, and we crafted each custom jaw with a laser cutter.  We considered using a mold and vacuum forming to create something a bit more custom, but had to scrap the idea due to time constraints.

Sensors

Our initial implementation for detecting punches failed due to far too many false positives. We thought using IR distance sensors would be a good solution since we could detect a “close” punch and tell the other robot to retract the arm before real contact. The test looked promising, but in practice, when the opposite sensors were close, we saw a lot of noise in the data. The backup and currently implemented solution was to install simple push switches in the chest and detect when those are clicked by the chest plate pressing against them.

Power

Different items required different voltages. The motors and pneumatic valves required 24V, the LEDs required 12V and the USB hub required 5V. We used Castle Pro BEC converters to step down the voltages. These devices are typically used in RC airplanes and helicopters.

Shipping

So how does someone ship two 700lb robots from Seattle to Austin? We did it in 8 crates. . The key thing to note is that the tops and bottoms of each robot were separated. Any wire that connected the two parts had to be able to be disconnected in some form. This affected the serial cords and the power cords (5V, 12V, 24V).

Software

The software and architecture went through a variety of iterations during development. The final architecture used 3 laptops, 2 desktops, an access point, and a router. It's important to note that the laptops of Robot 1 and Robot 2 are physically mounted on the backs of each Robot body, communicating through WiFi to the Admin console. The entire setup looks like the following diagram:

Admin Console

The heart of the infrastructure is the Admin Console. Originally, this was also intended to be a scoreboard to show audience members the current stats of the match, but as we got further into the project, we realized this wouldn't be necessary. The robots are where the action is, and people's eyes focus there. Additionally, the robots themselves display their current health status via LEDs, so duplicating this information isn't useful. However, the admin side of this app remains.

Sockets

The admin console is the master controller for the game state and utilizes socket communication between it, the robots, and the user consoles. A generic socket handler was written to span each computer in the setup. The SocketListener object allows for incoming connections to be received, while the SocketClient allows clients to connect to those SocketListeners. These are generic objects, which must specify objects of type GamePacket to send and receive:

public class SocketListener<TSend, TReceive> where TSend : GamePacket                                                
                                             where TReceive : GamePacket, new()

GamePacket is a base class from which specific packets inherit:

public abstract class GamePacket
{
    public byte[] ToByteArray() 
    {   
        MemoryStream ms = new MemoryStream();           
        BinaryWriter bw = new BinaryWriter(ms);
                
        try
        {           
            WritePacket(bw);
        }
        catch(IOException ex)
        {
            Debug.WriteLine("Error writing packet: " + ex);
        }
        
        return ms.ToArray();
    }
        
    public void FromBinaryReader(BinaryReader br)
    {        
        try
        {    
            ReadPacket(br);       
        }
        catch(IOException ex)       
        {
            Debug.WriteLine("Error reading packet: " + ex);
        }
    }
    
    public abstract void WritePacket(BinaryWriter bw);
    public abstract void ReadPacket(BinaryReader br);
}

For example, in communication between the robots and the admin console, GameStatePacket and MovementDescriptorPacket are sent and received. Each GamePacket must implement its own ReadPacket and WritePacket methods to serialize itself for sending across the socket.

Packets are sent between machines every "frame". We need the absolute latest game state, robot movement, etc. at all times to ensure the game is functional and responsive.

As is quite obvious, absolutely no effort was put into making the console "pretty". This is never seen by the end users and just needs to be functional. Once the robot software and the user consoles are started, the admin console initiates connections to each of those four machines. Each machine runs the SocketListener side of the socket code, while the Admin console creates four SocketClient objects to connect to each those. Once connected, the admin has control of the game and can start, stop, pause, and reset a match by sending the appropriate packets to everyone that is connected.

Robot

The robot UI is also never intended to be seen by an end user, and therefore contains only diagnostic information.

Each robot has a wireless Xbox 360 controller connected to it so it can be manually controlled. The UI above reflects the positions of the controller sticks and buttons. During a match, it's possible for a bot to get outside of our "safe zone". One bot might be pushing the other, or the user may be moving the bot toward the edge of the ring. To counter this, the player's coach can either temporarily move the bot, turning off Kinect input, or force the game into "referee mode" which pauses the entire match and turns off Kinect control on both sides. In either case, the robots can be driven with the controllers and reset to safe positions. Once both coaches signal that the robots are reset, the admin can then resume the match.

Controlling Hardware

Phidget hardware controlled our LEDs, relays, and sensors. Getting data out of a Phidget along with actions, such as opening and closing a relay, is shockingly easy as they have pretty straightforward C# APIs and samples, which is why they typically are our go-to product for projects like this.

Below are some code snippets for the LEDs, relays, and sensor.

LEDs – from LedController.cs

This is the code that actually updates the health LEDs in the robot's chest. The LEDs were put on the board in a certain order to allow this style of iteration. We had a small issue of running out of one color of LEDs so we used some super bright ones and had to reduce the power levels to the non-super bright LEDs to prevent possible damage:

private void UpdateLedsNonSuperBright(int amount, int offset, int brightness)
{
    for (var i = offset; i < amount + offset; i++)
    {
        _phidgetLed.leds[i] = brightness / 2;
    }
}

private void UpdateLedsSuperBright(int amount, int offset, int brightness)
{
    for (var i = offset; i < amount + offset; i++)
    {
        _phidgetLed.leds[i] = brightness;
    }
}

Sensor data – from SensorController.cs

This code snippet shows how we obtain the digital and analog inputs from the Phidget 8/8/8 interface board:

public SensorController(InterfaceKit phidgetInterfaceKit) : base(phidgetInterfaceKit)
{    
    PhidgetInterfaceKit.ratiometric = true;
}

public int PollAnalogInput(int index)
{
    return PhidgetInterfaceKit.sensors[index].Value;
}

public bool PollDigitalInput(int index)
{
    return PhidgetInterfaceKit.inputs[index];
}

Relays – from RelayController.cs

Electrical relays fire our pneumatic valves. These control the head popping and the arms punching. For our application, we wanted the ability to reset the relay automatically. When the relay is opened, an event is triggered and we create an actively polled thread to validate whether we should close the relay. The reason why we actively poll is someone could be quickly toggling the relay. We wouldn't want to close it on accident. The polling and logic does result in a possible delay or early trigger for closing the relay, but for the BoxingBots the difference of 10ms for a relay closing is acceptable:

public void Open(int index, int autoCloseDelay)
{   
    UseRelay(index, true, autoCloseDelay);
}

public void Close(int index)
{
    UseRelay(index, false, 0);
}

private void UseRelay(int index, bool openRelay, int autoCloseDelay)
{
    AlterTimeDelay(index, autoCloseDelay);
    PhidgetInterfaceKit.outputs[index] = openRelay;
}

void _relayController_OutputChange(object sender, OutputChangeEventArgs e)
{
    // closed
    if (!e.Value)
        return;

    ThreadPool.QueueUserWorkItem(state =>
    {                                          
        if (_timeDelays.ContainsKey(e.Index))
        {                                              
            while (_timeDelays[e.Index] > 0)                       
            {
                Thread.Sleep(ThreadTick);           
                _timeDelays[e.Index] -= ThreadTick;
            }                                            
        }
                                            
        Close(e.Index);
                                        
    });
}

public int GetTimeDelay(int index)
{
    if (!_timeDelays.ContainsKey(index))
        return 0;
    return _timeDelays[index];
}

public void AlterTimeDelay(int index, int autoCloseDelay)
{
    _timeDelays[index] = autoCloseDelay;
}

User Console

Since the theme of the party was Retro Gaming, we wanted to go for an early 80's Sci-fi style interface, complete with starscape background and solar flares! We wanted to create actual interactive elements, though, to maintain the green phosphor look of early monochrome monitors. Unlike traditional video games, however, the screens are designed not as the primary focus of attention, but rather to help calibrate the player before the round and provide secondary display data during the match. The player should primarily stay focused on the boxer during the match, so the interface is designed to sit under the players view line and serve as more of a dashboard during each match.

However, during calibration before each round, it is important to have the player understand how their core body will be used to drive the Robot base during each round. To do this, we needed to track an average of the joints that make up each fighter's body core. We handled the process by creating a list of core joints and a variable that normalizes the metric distances returned from the Kinect sensor into a human-acceptable range of motion:

private static List<JointType> coreJoints = newList<JointType>( 
    newJointType[] {
        JointType.AnkleLeft,
        JointType.AnkleRight,
        JointType.ShoulderCenter,
        JointType.HipCenter
    });
private const double RangeNormalizer = .22;
private const double NoiseClip = .05;

And then during each skeleton calculation called by the game loop, we average the core positions to determine the averages of the players as they relate to their playable ring boundary:

public staticMovementDescriptorPacket AnalyzeSkeleton(Skeleton skeleton)
{           
    // ...

    CoreAverageDelta.X = 0.0;
    CoreAverageDelta.Z = 0.0;
    foreach (JointType jt in CoreJoints)
    {
        CoreAverageDelta.X += skeleton.Joints[jt].Position.X - RingCenter.X;
        CoreAverageDelta.Z += skeleton.Joints[jt].Position.Z - RingCenter.Z;
    }

    CoreAverageDelta.X /= CoreJoints.Count * RangeNormalizer;
    CoreAverageDelta.Z /= CoreJoints.Count * RangeNormalizer;

    // ...

    if (CoreAverageDelta.Z > NoiseClip || CoreAverageDelta.Z < -NoiseClip)
    {
        packet.Move = -CoreAverageDelta.Z;
    }

    if (CoreAverageDelta.X > NoiseClip || CoreAverageDelta.X < -NoiseClip)
    {
        packet.Strafe = CoreAverageDelta.X;
    }
}

In this way, we filter out insignificant data noise and allow the player's average core body to serve as a joystick for driving the robot around. Allowing them to lean at any angle, the move and strafe values are accordingly set to allow for a full 360 degrees of movement freedom, while at the same time not allowing any one joint to unevenly influence their direction of motion.

Another snippet of code that may be of interest is the WPF3D rendering we used to visualize the skeleton. Since the Kinect returns joint data based off of a center point, it is relatively easy to wire up a working 3D model in WPF3D off of the skeleton data, and we do this in the ringAvatar.xaml control.

In the XAML, we simply need a basic Viewport3D with camera, lights, and an empty ModelVisual3D container to hold or squares. The empty container looks like this:              

<ModelVisual3D x:Name="viewportModelsContainer2">                   
    <ModelVisual3D.Transform>
        <Transform3DGroup>           
            <RotateTransform3D x:Name="bodyRotationCenter" CenterX="0" CenterY="0" CenterZ="0">      
                <RotateTransform3D.Rotation>             
                    <AxisAngleRotation3D x:Name="myAngleRotation" Axis="0,1,0" Angle="-40"/>                 
                </RotateTransform3D.Rotation>           
            </RotateTransform3D> 
        </Transform3DGroup> 
    </ModelVisual3D.Transform>
</ModelVisual3D>

In the code, we created a generic WPF3DModel that inherits from UIElement3D and is used to store the basic positioning properties of each square. In the constructor of the object, though, we can pass a reference key to a XAML file that defines the 3D mesh to use:

public WPF3DModel(string resourceKey)
{
    this.Visual3DModel = Application.Current.Resources[resourceKey] as Model3DGroup;
}

This is a handy trick when you need to do a fast WPF3D demo and require a certain level of flexibility. To create a 3D cube for each joint when ringAvatar is initialized, we simply do this:

private readonly List<WPF3DModel> _models = new List<WPF3DModel>();
private void CreateViewportModels()
{           
    for (int i = 0; i < 20; i++)
    { 
        WPF3DModel model = new WPF3DModel("mesh_cube");
        viewportModelsContainer2.Children.Add(model);

        // ...

        _models.Add(model);
    }

    // ...
}

And then each time we need to redraw the skeleton, we loop through the skeleton data and set the cube position like so:

if (SkeletonProcessor.RawSkeleton.TrackingState == SkeletonTrackingState.Tracked)
{
    int i = 0;  

    foreach (Joint joint in SkeletonProcessor.RawSkeleton.Joints)
    {
        if (joint.TrackingState == JointTrackingState.Tracked)
        {
                                        
            _models[i].Translate(                                        
                joint.Position.X * 8.0,                                    
                joint.Position.Y * 10.0,
                joint.Position.Z * -10.0);

            i++;
        }
    }

    // ...

}

There are a few other areas in the User Console that you may want to further dig into, including the weighting for handling a punch as well dynamically generating arcs based on the position of the fist to the shoulder. However, for this experience, the User Console serves as a secondary display to support the playing experience and gives both the player and audience a visual anchor for the game.

Making a 700lb Tank Drive like a First Person Shooter

The character in a first person shooter (FPS) video game has an X position, a Y position, and a rotation vector. On an Xbox controller, the left stick controls the X,Y position. Y is the throttle (forward and backward), X is the strafing amount (left and right), and the right thumb stick moves the camera to change what you're looking at (rotation).  When all three are combined, the character can do things such as run around someone while looking at them.

In the prior project, we had existing code that worked for controlling all 4 motors at the same time, working much like a tank does, so we only had throttle (forward and back) and strafing (left and right). Accordingly, we can move the motors in all directions, but there are still scenarios in which the wheels fight one another and the base won't move. By moving to a FPS style, we eliminate the ability to move the wheels in an non-productive way and actually make it a lot easier to drive.

Note that Clint had some wiring "quirks" with polarity and which motor was left VS right, he had to correct in these quirks in software :

public Speed CalculateSpeed(double throttleVector, double strafeVector, double rotationAngle)
{
    rotationAngle = VerifyLegalValues(rotationAngle);
    rotationAngle = AdjustValueForDeadzone(rotationAngle, AllowedRotationAngle, _negatedAllowedRotationAngle);

    // flipped wiring, easy fix is here  
    throttleVector *= -1;
    rotationAngle *= -1;
    
    // miss wired, had to flip throttle and straff for calc
    return CalculateSpeed(strafeVector + rotationAngle, throttleVector, strafeVector - rotationAngle, throttleVector);
}

protected Speed CalculateSpeed(double leftSideThrottle, double leftSideVectorMultiplier, double rightSideThrottle, double rightSideVectorMultiplier) 
{ /* code from Jellybean */ }

Conclusion

The Boxing Bots project was one of the biggest things we have built to date. It was also one of our most successful projects. Though it was a rainy, cold day and night in Austin when the bots were revealed, and we had to move locations several times during setup to ensure the bots and computers wouldn't be fried by the rain, they ran flawlessly for the entire event and contestants seemed to have a lot of fun driving them.

While the goal of these quickstarts is to discuss the issues and solutions involved in porting our Windows Phone 7 app to a Windows Store app, there are some features and requirements in Windows 8 we wanted to address: snapping and appcontracts.

Windows 8 Contracts provide OS level functionality and can significantly extend the power of your app when implemented properly. In this video, we will look at two contracts we implemented, the Search Contract and the Share Contract. But first, let us look at how snapping is handled in a Windows Store app.

Handling Window Snapping

As we discussed in the last Quickstart, snapping can have a significant impact on how you choose to present your app interface. In a typical template project page, we will find a GridView and ListView element both positioned in Grid.Row="1" under the navigation and page header row of our root grid.

It is the VisualStateManager that controls the visibility of these elements. Since PageHub.xaml inherits from  Common\LayoutAwarePage.cs, we already have the orientation and positioning handlers taken care of. When the app is "snapped" to the screen and has its visual footprint significantly reduced, the VisualStateManager will toggle the visibility of the main GridView to Collapsed while making the thinner ListView based interface visible as shown below:

One other thing to note in this discussion is that while the ListView and GridView are both showing the same data, they would most likely need to reference different data templates and styles.

In the Khan Academy app, the ListView control that handles the snapped layout shares the same CollectionViewSource as the larger GridView control, but points to its own ItemTemplateSelector so that it will pull in the snapped styles accordingly.

The Search Contract

The Search Contract allows your app to register with the operating system as being searchable from the Windows 8 Search charm.

To set this up, we need to add some code to the LaunchApp() method in App.xaml.cs as shown below:

App.xaml.cs

public async void LaunchApp(
    ApplicationExecutionState previousExecutionState)
{
    DataSource = new KhanDataSource();
    await DataSource.LoadAllData();
 
    SearchPane searchPane = SearchPane.GetForCurrentView();
    searchPane.PlaceholderText = “Search Khan Academy”;
    searchPane.QuerySubmitted +=
        (sender, queryArgs) =>
        {
            KhanAcademy.SearchResultsPage.Activate(
                queryArgs.QueryText, previousExecutionState);
        };
        […]

First, we get the SearchPane object for the current app. Once we have a handle to the instance of the pane, we can set some properties to make the search experience a bit more tailored to our app. Namely, we set the PlaceholderText property to the string “Search Khan Academy”. Finally, we hook the QuerySubmitted event off the SearchPane object, which will be called once the user presses the Enter key or clicks/taps the magnifying glass icon to the right of the search box. In the event handler, the SearchResultsPage is activated and displayed with the user’s query text passed to the page.

The Search pane can also handle real-time suggestions based on the query the user is typing. To do this, we just need to handle the SuggestionsRequested and ResultSuggestionChosen events as shown:

C#

            searchPane.SuggestionsRequested += 
                (sender, suggestionArgs) =>
                {
                    var videos = App.DataSource.TopicGroups.SelectMany(g =>
                        g.Playlists.SelectMany(p => p.Videos.Where(v =>
                            Regex.IsMatch(v.Name ?? "", suggestionArgs.QueryText, RegexOptions.IgnoreCase) ||
                            Regex.IsMatch(v.Description ?? "", suggestionArgs.QueryText, RegexOptions.IgnoreCase))))
                            .Distinct(VideoItem.CreateComparer())
                            .Take(3);
                    foreach(VideoItem vi in videos)
                        suggestionArgs.Request.SearchSuggestionCollection.AppendQuerySuggestion(vi.Title);
                    var recommended = App.DataSource.TopicGroups.SelectMany(g =>
                        g.Playlists.SelectMany(p => p.Videos.Where(v =>
                            Regex.IsMatch(v.Name ?? "", suggestionArgs.QueryText, RegexOptions.IgnoreCase)))).FirstOrDefault();
                    if(recommended != null)
                    {
                        suggestionArgs.Request.SearchSuggestionCollection.AppendSearchSeparator("Recommended");
                        IRandomAccessStreamReference imgStream = RandomAccessStreamReference.CreateFromUri(recommended.ImagePath);
                        suggestionArgs.Request.SearchSuggestionCollection.AppendResultSuggestion(recommended.Title, recommended.Description, recommended.VideoPath.ToString(), imgStream, recommended.Title);
                    }
                };
            searchPane.ResultSuggestionChosen +=
                (sender, resultArgs) =>
                {
                    var recommended = App.DataSource.TopicGroups.SelectMany(g =>
                        g.Playlists.SelectMany(p => p.Videos.Where(v =>
                            Regex.IsMatch(v.VideoPath.ToString() ?? "", resultArgs.Tag, RegexOptions.IgnoreCase)))).FirstOrDefault();
                    Frame f = Window.Current.Content as Frame;
                    f.Navigate(typeof(VideoPage), JsonSerializer.Serialize(recommended));
                };

In the SuggestionRequested event handler, we use a LINQ query to find the first 3 results that match what the user is typing and add them to the suggestions list using the AppendQuerySuggestion method off the SearchSuggestionCollection object. In addition, we also use LINQ to find the closest match to what the user is typing to display and display that after a line separator.

The line gets added with the AppendSearchSeparator method, and the single result gets added with the AppendResultSuggestion method. The specific result suggestion can include an icon, and we get the stream for that icon using the CreateFromUri method off the RandomAccessStreamReference object, passing in the path to the image we want to use as the icon. This returns the IRandomAccessStreamReference required by the AppendResultSuggestion method.

In the ResultSuggestionChosen method, we determine which suggestion the user selected and then navigate to the VideoPage to show the video to the user.

The Share and Devices Contracts

The Share Contract allows the user to share a piece of data from the current app to another app which knows how to receive the data. For our app, we wanted to allow the user to share a link to the video they are watching to other apps. This works great with the Windows 8 mail app.

To accomplish this, we override the DataRequested method on the VideoPage class as shown:

VideoPage.xaml.cs

public sealed partial class VideoPage
    : KhanAcademy.Common.LayoutAwarePage
{
    private PlayToManager _playToManager = null;
    private CoreDispatcher _dispatcher = null;
    private DisplayRequest _displayRequest = null;
 
    public VideoPage()
    {
        this.InitializeComponent();
    }
    protected override void DataRequested(
        DataTransferManager sender,
        DataRequestedEventArgs args)
    {
        VideoItem vi = (this.DataContext as VideoItem);
        args.Request.Data.Properties.Title = vi.Name;
        args.Request.Data.Properties.Description = vi.Description;
        args.Request.Data.SetUri(vi.KhanPath);
    }

In this method, we get the VideoItem that is currently being played, and use its properties to fill out the Data object property of the Request object. The SetUri method is used to pass the actual Uri to the Khan Academy website.

Finally, we wanted to enable PlayTo functionality so the user can quickly and easily send the video output to a TV or other media device on the network. The user can choose to do this while watching the video by selecting the Devices charm. Any PlayTo-enabled devices on the network will show up in this device list.

In code, we handle this scenario in the OnNavigatedTo and OnNavigatedFrom overridden methods.

VideoPage.xaml.cs

protected override void OnNavigatedTo(NavigationEventArgs e)
{
    _dispatcher = Window.Current.CoreWindow.Dispatcher;
    _playToManager = PlayToManager.GetForCurrentView();
    _playToManager.SourceRequested += playToManager_SourceRequested;
    if(_displayRequest == null)
        _displayRequest = new DisplayRequest();
    _displayRequest.RequestActive();
    base.OnNavigatedTo(e);
}
protected override void OnNavigatedFrom(NavigationEventArgs e)
{
    _playToManager.SourceRequested -= playToManager_SourceRequested;
    if(_displayRequest != null)
        _displayRequest.RequestRelease();
    base.OnNavigatedFrom(e);
}
void playToManager_SourceRequested(PlayToManager sender, PlayToSourceRequestedEventArgs args)
{
    var deferral = args.SourceRequest.GetDeferral();
    var handler = _dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
    {
        args.SourceRequest.SetSource(videoElement.PlayToSource);
        deferral.Complete();
    });
}

In the OnNavigatedTo method, we get an instance of the PlayToManager and hook the SourceRequested event. We also grab an instance of the current Dispatcher. In the SourceRequested event handler, we request a deferral. This tells the PlayTo manager to wait until we give it the PlayTo source so the request doesn’t time out. We then, using the dispatcher, set the source of the PlayTo request to the video player’s PlayToSource property, and then complete the deferral. At this point, the PlayTo request is sent and the media player talks to the OS to make the magic happen. Note that in the OnNavigatedFrom method we unhook the SourceRequested event handler since it is no longer needed once the user navigates away from this page.

So at this point, we have come up with a migration strategy for porting our Windows Phone 8 to Windows 8, handled changes to how we handle asynchronous data requests and have a well styled, data bound interface that leverages the newest features of Windows 8. From here, the next step would be submitting to the store for distribution.

Hopefully, this set of quickstarts has covered some key areas of interest in getting your app up and running on Windows 8.

I would like to extend a tremendous amount of gratitude for Joel Martinez and all his tireless work as well as opening up the Viewer for Khan Academy app to serve as a Coding4Fun Community project here on Channel 9.

The source code for the Windows 8 Khan Academy app is available for download:

If you have any questions, comments, or feedback feel free to join in the discussion.

Twitter: @rickbarraza, @joelmartinez

For example, if you don’t know how to copy a file in JavaScript or C++, just right-click, select Insert Snippet, select the code snippet, and it will automatically add the code to your Windows app as shown in the screenshots below.

Download It

Download Now

Please see the following pages to learn how to install the snippets and how to use them.

• Installation
• See the full list of Snippets

Screenshots

After installing, right-click and select Insert Snippet...and select the snippet from the list. Then press Tab to jump between fields.

Feedback

Give them a try and let us know what you think!

• [0:51] http://www.buildwindows.com/, Build 2012 is sold out. Tune in online., http://channel9.msdn.com/Events/BUILD/2012
• [1:50] What is Portable Class Library project in Visual Studio 2012– .NET 4.5 (Kishore Babu Gaddam)
• [3:04] Javascript Fundamentals: Development for Absolute Beginners (Clint Rutkas, Bob Tabor, Golnaz )
• [3:54] Building your own Windows Runtime components to deliver great Metro style apps (Ines Khelifi)
• [6:22] UPDATED HOWTO: SQLite with Windows 8 apps (Tim Heuer)
• [7:34] Maximizing Code Reuse Across Windows 8 and Windows Phone Apps (Alnur Ismail)
• [8:37] Callisto (Tim Heuer) [Found via: CALLISTO Library]
• [9:39] Large collection of Free Microsoft eBooks for you, including: SharePoint, Visual Studio, Windows Phone, Windows 8, Office 365, Office 2010, SQL Server 2012, Azure, and more. , Another large collection of Free Microsoft eBooks and Resource Kits for you, including: SharePoint 2013, Office 2013, Office 365, Duet 2.0, Azure, Cloud, Windows Phone, Lync, Dynamics CRM, and more. (Eric Ligman), Microsoft Technologies E-Book Gallery (TechNet Wiki)
• [10:32] Channel 9 Highlight: Defrag Tools (Andrew Richards, Larry Larsen)

Picks of the Week!

• Brian's Pick of the Week:[11:21] KinÊtre - Animating the World with the Human Body (Steve Clayton)
• Dan's Pick of the Week: [12:58] Build Easter Egg http://www.buildwindows.com 

 
Photo courtesy of John Boxall

Introduction

When the Arduino Duemilanove microcontroller appeared in 2005, it featured a set of female pin headers exposing most of the pins of the ATmega168 for easy hacking and for connecting accessory boards known as 'Shields'. The purpose of a shield is to provide new plug-and-play functionality to the host microcontroller, such as circuit prototyping, motion control, sensor integration, network and radio communication, or gaming interfaces, without worrying too much about the hardware implementation details. Seven years after the birth of the original Arduino, new shields keep coming out and are being cataloged on http://shieldlist.org/, a testament to the versatility of the design. It is also simple to build a DIY shield when nothing out there will meet your needs or when you want to understand how the shield concept works from the ground up.

In their infinite wisdom, Secret Labs, the makers of the Netduino, adopted the same Arduino form factor and header pin out to let .NET Micro Framework users tap into the vast array of Arduino shields on the market. Yet, there's always one hurdle that needs to be overcome: in order to give access to their hardware functions easily, many Arduino shields provide sample sketches (a.k.a. C/C++ Wiring programs) and libraries also written in C/C++ designed to be used within the Arduino IDE. Unfortunately, Arduino sketches and libraries aren't compatible with the Visual Studio environment and the .NET runtime: they need to be adapted in order to make use of the shield's hardware.

The goal of this series of articles is to attempt to demystify what an Arduino shield is. To do this, we'll start by building one from scratch. Then, we'll present a generic approach for selecting Netduino-compatible Arduino shields and adapting Arduino code to interface with the .NET Micro Framework.

Selecting Arduino shields for use with Netduino / Plus

Arduino shields using hardware components interfaced over protocols such as SPI, I2C, 1-Wire or serially over a UART are generally good candidates for a Netduino adaptation because the .NET Micro Framework supports these protocols natively and can do bulk I/O transfers, yielding good performance.

However, if a shield requires bit-banging data on GPIOs (i.e. communicating data serially, such as SPI, I2C, RS-232, in software by toggling GPIO lines individually instead of using dedicated hardware interfaces) or has very low I/O latency requirements, it may not be practical or even possible to use the shield on a Netduino / Plus.

The reason for this is simple: the Arduino runs native code and can access hardware registers directly with a latency orders of magnitude lower than what the .NET Micro Framework can achieve on a Netduino / Plus where toggling GPIOs means going through multiple framework layers while running interpreted code. The maximum raw I/O toggling frequency on a Netduino / Plus clocked at 48 MHz was measured around ~8.4KHz which is slow compared to the few MHz achievable on an Arduino clocked at 16 MHz.

Fortunately, there are possible workarounds and strategies

• Whenever an Arduino library bit-bangs SPI data on GPIOs, regardless of the pins used to do it, it can be replaced by using the standard SPI interface on the Netduino / Plus.
• The Netduino / Plus has significantly more RAM available than the Arduino and can transfer larger SPI data buffer in a single shot, dramatically reducing I/O latency. For instance, driving a display shield would greatly benefit from this method where a frame is cached in RAM on the Netduino / Plus then sent in one shot over SPI to the hardware display driver.

The bottom line is that it is always a good idea to review how the Arduino code interfaces with a given shield before proceeding with a purchase or starting a conversion project. This review process is also facilitated by http://shieldlist.org which shows which pins are being used to drive a shield. For example, the Adafruit Industries Logger Shield uses hardware SPI (pins D10-D13) for the SD card and I2C (pins A4-A5) for the DS1307 real time clock, making it a perfect adaptation candidate for the Netduino / Plus as we'll see in our next article.

It's also very important to know what features are available on the .NET Micro Framework running on the Netduino / Plus: some things come standard with the .NET Micro Framework which normally require add-ons libraries on the Arduino such as file system and networking support.

Finally, it is wise to search code repositories such as CodePlex, BitBucket and GitHub for hardware drivers and libraries before undertaking writing one in C# from scratch based on component datasheets. This extends to the Netduino and the TinyCLR community forums where many people have posted drivers in discussion threads.

Building a shield from scratch to meet specific requirements

Recently, we were asked if it was possible to connect our Touch Display module designed for a Netduino Go! to a Netduino / Plus. By doing a bit of research and seeing how the driver for the Touch Module works, this can be done by building a simple custom shield composed of a few parts connecting the Netduino pin headers to a Netduino Go! socket breakout board like so:

High-Level Software Interface

To frame the rest of this article, the communication interface with the shield is done using SPI and a GPIO used for synchronization. The communication protocol details are wrapped in the Nwazet.Go.Display.Imaging.VirtualCanvas class which needs to be initialized with the SPI and GPIO parameters matching the pin out of the shield:

var canvas = new VirtualCanvas(TouchEventHandler, WidgetClickedHandler);

canvas.Initialize(
    displaySpi: SPI.SPI_module.SPI1,
    displayChipSelect: Pins.GPIO_PIN_D9,
    displayGPIO: Pins.GPIO_PIN_D8,
    speedKHz: 5000);

From that point on, all interactions with the shield are done through Draw* methods followed by a call to the Execute() method which handles sending and receiving SPI data packets. The VirtualCanvas class also provides methods for creating widgets such as buttons, receiving touch screen events and setting the screen orientation.

For example, the following code snippet draws and updates a progress bar and uses one of the proportional fonts built into the display:

public static void Main() {
    var canvas = new VirtualCanvas();
    canvas.Initialize(
        displaySpi: SPI.SPI_module.SPI1,
        displayChipSelect: Pins.GPIO_PIN_D9,
        displayGPIO: Pins.GPIO_PIN_D8,
        speedKHz: 5000);
    var fontInfo = new DejaVuSans9().GetFontInfo();
    canvas.DrawFill(0xFFFF);
    canvas.DrawString(5, 144, (ushort)BasicColor.Black, fontInfo.ID, "Progress");
    for (var progress = 1; progress <= 100; progress++) {
        canvas.DrawProgressBar(
            70, 140,
            75, 12,
            RoundedCornerStyle.All,
            RoundedCornerStyle.All,
            (ushort)BasicColor.Black,
            (ushort)GrayScaleValues.Gray_128,
            (ushort)GrayScaleValues.Gray_30,
            (ushort)BasicColor.Green,
            progress);
        Thread.Sleep(100);
        canvas.Execute();
    }
}

Netduino Go! Socket Pin Out

In order to build the shield, we need to start with mapping out the connections between the Go! bus socket and the header pins of the shield.

The Go! socket pin out was designed by Secret Labs to be compatible with Gadgeteer modules supporting the following socket types: S (SPI), U (UART), X (3 GPIO) as documented in the .NET Gadgeteer Mainboard Builder's Guide v 1.8, page 6. This makes our little shield compatible with a Gadgeteer board as long as it is connected to an S socket. As far as other .NET Micro Framework boards are concerned, they should also work fine if they can drive an SPI slave device at 5 MHz, the minimum SPI clock frequency required by the Touch Display module.

Gadgeteer Socket Types

Netduino Go! Socket

The hardware specifications of the Netduino / Plus tells us that the Netduino GPIOs correspond exactly to the Arduino Uno GPIOs:

• digital pins 0-1: UART 1 RX, TX
• digital pins 2-3: UART 2 RX, TX
• digital pins 5-6: PWM, PWM
• digital pins 7-8: UART 2 RTS, CTS
• digital pins 9-10: PWM, PWM
• digital pins 11-13: SPI MOSI, MISO, SPCK
• analog pins 4-5: I2C SDA, SCL

With this information, we can proceed with the creation of schematics to bridge the gap between the two worlds.

Shield schematics

How does this custom shield work?

The Netduino Go! bus is designed around the SPI protocol (see Go! socket pins 6-10) which the Netduino / Plus can speak easily, making a Netduino Go! module or a S-type Gadgeteer module appear like any other SPI device to a Netduino / Plus.

You will notice that the following pins aren't connected on the Go! socket for the sake of simplification:

• Pin 2 (5v power): the Touch Display module operates on 3.3v, therefore the 5v power supply is not needed but could be connected to the 5v header of the Netduino to be fully compliant.
• Pin 4 and 5 (UART): the Touch Display module does not make use of the serial interface during normal operation, so these lines can remain disconnected. However, to be fully compliant, Pin 4 (RX) and 5 (TX) on the Go! socket could be respectively connected to Netduino pin D1 (TX) and D0 (RX).
• The fact that these lines aren't connected in this specific scenario would cause another module requiring a UART and/or 5v to not work obviously.

While the SPI part is straight forward, there's a bit more to the Go! bus electrical specification that must be taken into account: the Netduino Go! bus also uses its GPIO pin on boot (see Go! socket pin 3) to control the behavior of modules based on ARM Cortex chips: when the GPIO pin is held HIGH on module power-up, the ARM Cortex chip enters its bootloader mode and waits to be flashed with new firmware over the Go! bus UART (see Go! socket pins 4-5). The ARM Cortex bootloader remains active until the ARM Cortex chip is reset or power-cycled.

This is important to know for two reasons:

1. The Touch Display module is powered by an ARM Cortex M3 chip.
2. On boot, a Netduino / Plus sets all of its I/O pins HIGH, preventing the Touch Display from booting its firmware, which is undesirable. The reason for the I/O pins being set HIGH is documented on page 5 of the ATMEL AT91SAM7X datasheet which states that the pins on Parallel IO Controller A and B (PA0 - PA30 and PB0 - PB30) are pulled-up input at reset. The pins stay in that HIGH state until they're reconfigured by the Tiny CLR and the C# application itself.

For this reason, the custom shield is designed to control the power supply of the Touch Display module through a general-purpose 2N4403 PNP transistor (see T1_PNP on the shield schematics) connected to pin D7: it is only when the base of the transistor is asserted LOW that power flows to the Touch Display module, countering the behavior of the Netduino / Plus on boot. In addition, the Netduino / Plus controls the state of the Go! bus GPIO pin through D8, asserting it LOW before powering up the Touch Display module.

About the 2N4403 transistor: any other PNP transistor, or a P-Channel MOSFET, capable of switching at least 150mA continuously, the maximum current consumption of the touch display with some headroom, would have worked equally as well. The 2N4403 used here is more than adequate:

In the context of this shield, the PNP transistor is used as a switch, either fully ON or OFF, as opposed to being used as an amplifier. To this end, a 1K ohm resistor is used between the base of the transistor and Netduino / Plus pin D7 (POWER_CTRL) to ensure that the transistor is fully saturated but also limiting the maximum current sunken through pin D7 and potential damage being done to the ATMEL chip. The 10K ohm pull-up resistor placed between the base of the transistor and the 3.3v power rail ensures that the transistor will always be turned off by default, even when pin D7 is floating (i.e. not specifically declared as an Input or as an Output and driven HIGH or LOW by the C# application).

The PowerUpDisplay function summarizes the interaction with the module through the shield on boot:

// Pin connected to the transistor controlling the display's power supply
public static OutputPort PowerTransistor = new OutputPort(Pins.GPIO_PIN_D7, true);

public static void PowerUpDisplay() {
    // Ensure that the GPIO pin in low to prevent the display module to start in bootloader mode
    var goBusGPIO = new OutputPort(Pins.GPIO_PIN_D8, false);
    // Power up the display module
    PowerTransistor.Write(false);
    // Always wait 250ms after power-up to ensure that the display module is fully initialized before sending commands
    Thread.Sleep(250);
    goBusGPIO.Dispose();
}

Refactoring a Netduino Go! C# driver to work on a regular Netduino / Plus

The final step to get the Touch Display module working outside of its natural environment involved refactoring its C# driver (VirtualCanvas.cs), eliminating the idiosyncrasies specific to Netduino Go! bus. Go! modules are supposed to implement the GoModule interface as defined by the Netduino Go! SDK. Because the regular Netduino / Plus SDK has no knowledge of this interface, it's necessary to strip out any references and API calls specific to the Go! framework, leaving only what's absolutely required to communicate with the Touch Display module over SPI and maintaining the disposable nature of the class. For more details on Go! SDK, please refer to the Netduino Go! community forums and the source code of the existing module drivers as it is currently the best source of documentation.

This step is akin to the work required when adapting an Arduino library to run on the .Net Micro Framework, only in this case, the task is easier since we're dealing with a C# driver to begin with instead of going from C/C++ to C#.

Starting from the top of VirtualCanvas.cs, the following changes were made in the Netduino / Plus version of the driver:

• Removed
  • using GoBus;
• Replaced
  • public class VirtualCanvas : GoModule
  • with: public class VirtualCanvas : IDisposable
• Replaced
  • public void Initialize(GoSocket socket, uint speedKHz = 25000)
  • with: public void Initialize(SPI.SPI_module displaySpi, Cpu.Pin displayChipSelect, Cpu.Pin displayGPIO, uint speedKHz = 25000). The core of the function simply initializes the SPI interface.
• Replaced
  • protected override void Dispose(bool disposing)
  • with: public void Dispose() also removing the following calls:
    • SetSocketPowerState(false); this call is used by the GoModule object model to control when a socket is powered ON / OFF and is no longer applicable when the VirtualCanvas class is not derived from GoModule.
    • base.Dispose(disposing); by the same token, deriving from GoModule requires overriding the behavior of the Dispose function and calling the base class at the end, something that is no longer needed now that the class implements the regular IDisposable interface.
• Moved
  • GoBusIrqEvent.Reset(); from WaitUntilGoBusIrqIsAsserted()
  • to: Execute(Synchronicity sync = Synchronicity.Synchronous) taking into account the higher latency of the Netduino / Plus

The rest of the code remained unchanged. The Netduino / Plus Touch Display driver and test application for this custom shield can be downloaded from the Netudino Helpers project on CodePlex in the \Samples\Coding4Fun directory.

Conclusion

With nearly 300 known Arduino shields on the market, there's a wide variety of plug-and-play functionality waiting to be leveraged by the Netduino community, sometime with little or no software work required. When all else fails, building a shield from scratch to get a desired feature onto a Netduino / Plus is a relatively simple process only requiring reading component datasheets and some patience with a soldering iron or a small breadboard. In our next article, we'll put these guidelines to the test with a simple and popular off-the-shelf data logger shield made by Adafruit Industries.

Appendix

To build the shield interfacing with the [nwazet Touch Display, you will need the following parts:

Custom shield Eagle schematics

• Custom shield

Bill of materials (~$15 for the shield w/o the [nwazet Touch Display module)

• 1 [nwazet Touch Display module
• 1 10cm FCI connection cable
• 1 Adafruit DIY shield for Arduino
• 1 ProtoAdvantage IDC Cable Breakout Board
• 1 PNP Transistor 2N4403
• 1 1K ohm resistor
• 1 10K ohm resistor
• 1 22 AWG solid core wire
• 1 0.32" diameter 60/40 rosin-core solder

Eagle libraries

• Adafruit Eagle library (Netduino shield part)
• [nwazet Eagle library (Go Socket part)

Netduino Go! Wiki

• Netduino GO! Module Builder's Guide (documents the Go! bus specifications)

Bio

Fabien is the Chief Hacker and co-founder of Nwazet, a start-up company located in Redmond WA, specializing in Open Source software and embedded hardware design. Fabien's passion for technology started 30 years ago, creating video games for fun and for profit. He went on working on mainframes, industrial manufacturing systems, mobile and web applications. Before Nwazet, Fabien worked at MSFT for eight years in Windows Core Security, Windows Core Networking and Xbox. During downtime, Fabien enjoys shooting zombies and watching sci-fi.

At TechEd North America (June 11-14), you can stop by and check out the Project Detroit Mustang, or battle against another attendee with our BoxingBots!

Project Detroit will be on the show floor Sunday through Wednesday.  Stop by and get a picture of yourself in the driver's seat and try out all the features this car has to offer.

We will be running boxing matches with the BoxingBots on Monday night during the Welcome Reception, and Tuesday through Thursday around lunchtime.  Battle against a fellow attendee after fueling up!

If you're heading to TechEd Europe (June 26-29), we will only have the BoxingBots with us.  Apparently getting the car to Amsterdam is a bit tricky.  Step up to battle another attendee on Tuesday night during the Welcome Reception, or Wednesday through Friday around lunchtime.

In addition, at both conferences, we will be giving a session on how to create fantastic Kinect applications for Windows, and we'll have some very fun demos to show. Here are the details:

Coding4Fun: Build Fun, Cool, Commercial Applications Using the Kinect for Windows SDK
Session Code: DEV330

Speakers (North America): Brian Peek, Dan Fernandez
Speakers (Europe): Clint Rutkas, Dan Fernandez

When (North America): Tuesday, June 12 at 5:00 PM - 6:15 PM in S320A
When (Europe): Wednesday, June 27 at 5:00 PM - 6:15 PM

Come to this must-see session to find out how you can use the Kinect for Windows SDK to build commercial applications! Learn how the RGB and depth cameras work, how to use and fine-tune skeletal tracking, how to read an audio stream from the Kinect microphone array and even how to send Kinect data over the network including to your Windows Phone

So, stop by, say hello, and have some fun with a couple of our projects.  Hope to see you all there!

Overview

The external lighting system for Project Detroit is run on a web server on a Netduino Plus, which uses the .NET Micro Framework (NETMF). To get an accelerated start, we leveraged code from the CodePlex project Netduino Helpers to control the AdaFruit LPD8806 LED strip, and the web server base code from the Netduino forums. They have been tweaked slightly for our project.

Controllers, Routes, RESTful oh my!

We wanted the NETMF web server to have the exact same routes as the full-blown REST service layer in Detroit. With compiler conditionals, this allowed the same code to use the same routes on both the full .NET stack and NETMF. Since we couldn't use the ASP.NET MVC framework here, we had to make something close to it.

Controllers

Controllers implement an interface with a method named ExecuteAction. When GetController is called, we return the correct controller but cast as the interface IController. This lets us figure out what controller should be executing the request and continue to work generically in the context of the request connection thread. If new functionality is needed, we create a new controller that inherits from IController and update GetController:

private static void RequestReceived(Request request)
{
    // url comes in as /foo/bar/12345?style=255
    //
     // /CONTROLLER/ACTION/
    //
    // LIGHTING SAMPLE URL: /0/1/?r=255&g=255&b=255&zone=EnumAsInt
    // /0/... 0 = NetduinoControllerType.Lighting
    var validResponse = false;
    try
    {
        Logger.WriteLine("Start: " + request.Url + " at " + DateTime.Now);
        var urlInParts = request.Url.TrimStart(UriPathSeparator).Split(new[] {'?'}, 2);
        string[] uriSubParts = null;
        string[] queryStringSubParts = null;
        if (urlInParts.Length > 0)
            uriSubParts = urlInParts[0].Split(UriPathSeparator);
                
        if (urlInParts.Length > 1)
            queryStringSubParts = urlInParts[1].Split(QueryStringSeparator);
        if (uriSubParts != null && uriSubParts.Length > 1)
        {
            var targetController = GetController(uriSubParts[0]);
            if (targetController != null)
            {
                var action = (uriSubParts.Length >= 2) ? uriSubParts[1] : string.Empty;
                var result = targetController.ExecuteAction(action, queryStringSubParts);
                validResponse = true;
                request.SendResponse(result.ToString());
                Logger.WriteLine("Result: " + result + " from " + request.Url + " at " + DateTime.Now);
            }
        }
        if (!validResponse)
        {
            SendIndexHtml(request);
        }
    }
    catch(Exception ex0)
    {
        Logger.WriteLine(ex0.ToString());
    }
}

Routes, strings and enums on NETMF

NETMF can't parse an enum from a string, which is something that can be done in the full .NET Framework. But what does this actually mean? It means the routes for the NETMF web server are a bit unreadable. Here is the same route but with the parsing issue exposed:

Detroit's REST service route with ASP.NET MVC:
http://…/Lighting/Solid/?zone=all&r=255&g=0&b=0

Detroit's NETMF route:
http://…/0/3/?zone=0&r=255&g=0&b=0

This issue made direct device debugging a bit harder.

Parsing the Action and query string

Parsing the route is broken into two parts. The first is determines what controller to target. For Project Detroit, that meant the lighting system or the rear glass system. Once we know the controller, we still need to parse the query string and action into something more useful:

#if NETMF
private static readonly char[] QueryStringParamSeparator = new[] { '=' };
public static LightingData ParseQueryStringValues(string action, params string[] queryStringParameters)
#else
public static LightingData ParseQueryStringValues(string action, Dictionary<string, object> queryStringParameters)
#endif
{
    var data = new LightingData {LightingZone = LightingZoneType.All, LightingAction = ParseLightingActionType(action)};
#if NETMF
    if (queryStringParameters != null)
    {
        foreach (var queryString in queryStringParameters)
        {
            var valueKey = queryString.Split(QueryStringParamSeparator);
            if (valueKey.Length != 2)
                continue;
            var key = valueKey[0];
            var value = valueKey[1];
#else
        foreach(var key in queryStringParameters.Keys)
        {
            var value = queryStringParameters[key].ToString();
#endif
            switch (key.ToLower())
            {
                case ZoneHuman:
                case ZoneComputer:
                    data.LightingZone = ParseLightingZoneType(value);
                    break;
                // more stuff
            }
        }
#if NETMF
    }
#endif
    // more stuff
            
    return data;
}

Turning on the lights

Individually addressable LED strips allow us to tell each individual LED what color to be. To adjust the colors of the LED lights, we loop through all the LEDs, set their color value, and then call LedRefresh() on our LED strip. There's no need to set all of them, only the ones being updated:

private static void SolidAnimationWorker()
{
    var leds = GetLedsToIlluminate();
    var dataCopy = _data;
    for (var i = 0; i < leds.Length; i++)
        SetLed(leds[i], dataCopy.Red, dataCopy.Green, dataCopy.Blue);
    LedRefresh();
}

Zones / Animation Threads

A solid color is boring, but we can use procedural animations to add some flair. The current code supports the following animations:

• Solid - A single solid color.
• Fill - Goes from black to the desired color. The color stays lit. Much like filling a glass of liquid.
• Random - LEDs are set to a random value.
• Pulse - A glowing effect from bright to black.
• Snake - A one way lighting effect with a fading tail that moves left to right.
• Sweep - This effect was inspired by KITT and a Cylon robot and is similar to a snake pattern. The effect can also handle being split in two while maintaining a continual line for items such as the grill or the wheels.
• Police - Each side flashes a red and blue light.

Since we know the LED strip is a single strand, we can be clever with offsets and create groupings. By having a dedicated animation thread in each zone, we can have the grill of the car in Police Mode while the bottom fades to red and the vents fill to blue at different refresh rates.

To run an animation, we need to first kill the old animation thread, spin up a new thread, mark LEDs in the "All LED" zone as do not touch, and execute the new pattern. Without a way to mark the LEDs as "bad" in the "All" zone, the two animation threads would compete for setting the color and result in a flickering effect. Other zones don't have to worry about this since there is zero overlap of LEDs: 

public bool ExecuteAction(string action, params string[] queryStringParameters)
{
    var data = LightingDataHelper.ParseQueryStringValues(action, queryStringParameters);
            
    SignalToEndAnimationThread(data.LightingZone);
    int counter = 0;
    while (counter++ < 5 && IsThreadAlive(data.LightingZone))
        Thread.Sleep(10);
    if (IsThreadAlive(data.LightingZone))
        AbortAnimationThread(data.LightingZone);
    // moving data bucket into public so new threads can leverage it
    _data = data;
    if (_data.LightingAction == LightingActionType.Police || _data.LightingAction == LightingActionType.Sweep)
    {
        _data.LightingZone = LightingZoneType.Grill;
        if (IsThreadAlive(_data.LightingZone))
            AbortAnimationThread(_data.LightingZone);
    }
    // cleaning up just incase
    if (data.LightingZone != LightingZoneType.All)
    {
        for (var i = 0; i < _zoneAllAnimation.Length; i++)
        {
            var leds = GetLedsToIlluminate();
            for (var z = 0; z < leds.Length; z++)
            {
                // flagging item as bad in animation lib
                if (_zoneAllAnimation[i] == leds[z])
                    _zoneAllAnimation[i] = -1;
            }
        }
    }
    switch (_data.LightingAction)
    {
        case LightingActionType.Solid:
            Debug.Print("Solid");
            ExecuteThread(SolidAnimationWorker);
            break;
        // more patterns
    }
    return true;
}

We can now do something more interesting, like Police Mode:

private static void PoliceModeAnimationWorker()
{
    var leds = GetLedsToIlluminate();
    var dataCopy = _data;
    var length = leds.Length;
    var half = length / 2;
    var index = 0;
    while (IsThreadSignaledToBeAlive(dataCopy.LightingZone))
    {
        int redIntensity = (index == 0) ? 255 : 0;
        int blueIntensity = (index == 0) ? 0 : 255;
        for (var i = 0; i < half; i++)
            SetLed(leds[i], redIntensity, 0, 0);
        for (var i = half; i < length; i++)
            SetLed(leds[i], 0, 0, blueIntensity);
        LedRefresh();
        index++;
        index %= 2;
        Thread.Sleep(100);
    }
}

Improvements

Hindsight is 20/20. Nothing is ever perfect and there are of course items we'd love to change. At the start, the goal was to make this fairly generic and not Project Detroit "aware." As you can see by looking at the source, that wasn't accomplished due to time constraints.

We would use a different LED system for two reasons: data transfer speed and power. The longer the LED strip is, the longer it takes to address an item further down the line. Project Detroit, having 15 meters of lighting in it, was impacted by this issue. It works well, just not as well as we hoped. The voltage requirements of the LEDs are another consideration. They are 5V, while a car uses 12V. Had we used 12V LEDs, our wiring would have been far simpler and wouldn't have required multiple transformers to get the required amperage needed at the correct voltage.

Conclusion

We think the lighting solution, coupled with the time and material constraints, worked out pretty well.  Project Detroit was a ton of fun to build out and working with West Coast Customs was the chance of a lifetime.

Key Design Decisions

It’s important to keep in mind that this car was built for a TV show with a set schedule. As a result, there are a number of unique design decisions that came into play.

Schedule

Working backwards, the reveal for the car was set for Monday November 28, 2011 at the Microsoft Store in Bellevue, Washington. We started the project in early August, which gave us approximately 12 weeks for research, development, vehicle assembly, and testing. This was by far the #1 design decision as any ideas or features for the car had to be implemented by the reveal date.

Off the Shelf Parts

Another key design decision was to, where possible, use off-the-shelf hardware and software in order to allow  interested developers to build and reuse some of the subsystems for their own car (at least the ones that don’t require welding). For example, instead of buying pricey custom sized displays for the instrument cluster or passenger display, we used stock Samsung Series 7 Slate PCs and had West Coast Customs do the hard work of building a custom dash to hold the PC.

Hardware and Networking

The car is packed with a variety of computers and networking hardware.

• Instrument Cluster Slate – This slate is on the driver's side and manages the instrument cluster application and the On-Board Diagnostic (OBD) connection to read telemetry data from the car.
• Passenger Slate – This slate, which is built into the passenger's side, runs a custom Windows 8 application (see Passenger slate below).
  
• Laptop 1 – This laptop runs the REST service to control different parts of the car, the Kinect socket service for the front Kinect, and the user message service to display messages on the rear glass while driving.
• Laptop 2 – This laptop runs the Heads Up Display (HUD) service, the Kinect socket service for the back Kinect, the OBD-II database, and Azure services.
• Windows Phone – A Nokia Lumia 800 connects via WiFi and a custom Windows Phone 7 application (See Windows Phone application below).
• Xbox 360 – The Xbox 360 displays on either the passenger HUD or the rear glass display.
• Networking – A NETGEAR N600/WNDR3700 wireless router provides wired and wireless access for everything in the car, which is used in conjunction with a Verizon USB network card plugged into a Cradle Point MBR900 to provide an always-on 3G/4G LTE internet connection. The slates, laptops, and Xbox 360 are connected via CAT5e cable, while the Windows Phone 7 connects via WiFi.

Note: One of the limitations of the Kinect SDK is that if you have multiple Kinects plugged into one PC, only one of those Kinects can do skeletal tracking at a time (color/depth data works just fine). Because of this, we decided to have a dedicated laptop plugged into the front Kinect and another laptop plugged into the back Kinect in order to allow front and back skeletal tracking at the same time. If we'd not used simultaneous skeletal tracking, we could have combined all of the systems onto a single laptop.

Architecture

Here is a quick overview of the application architecture.

REST Service Layer

The REST Service Layer allowed different systems talk to one another. More importantly, it allowed different services to control hardware they normally wouldn't be able to access.

1. Thin client approach
   The solution we chose was to have all the services that control different parts of the car reside on the laptops and have client applications like the Windows Phone application send REST commands to execute an action so the service layer would execute the request.
   
   
2. REST-enable hardware
   Controlling hardware should be invisible to the consuming clients. For example, hardware that requires USB communication would be impossible to control with a Windows Phone. The service layer allowed us to control hardware in a way that was invisible to the end user.
   
   
3. Helper Libraries
   To simplify communication with the service layer, we built a set of helper classes to abstract out repetitive tasks like JSON serialization/deserialization, URI building, etc. For example, to get the list of car horn “ringtones”, the client application can call HornClient.List() to get back a list of available ringtone filenames. To set the car horn, the client calls HornClient.Set(filename), and to play the car horn, it then calls HornClient.Play(filename). The Helper libraries were built to work on Windows 7, Windows 8, and Windows Phone 7.

OBD-II

We have already released an article and library on the OBD-II portion of the car.  In short, OBD-II stands for On-Board Diagnostics. Hooking into this port allows one to query for different types of data from the car, which we use to get the current speed, RPMs, fuel level, etc. for display in the Instrument Cluster and other locations. OBD can do far more than this, but it's all we needed for our project. Please see the linked articles for further details on the OBD-II library itself.

For the car, because only one application can open and communicate with a serial port at one time, we created a WCF service that polls the OBD-II data from the car and GPS data from a Microsoft Streets & Trips GPS locator, and returns it to any application that queries the service.

For the OBD library, we used a manual connection to poll different values at different intervals. For values critical to driving the car—like RPM, speed, etc.—we polled for the values as quickly as the car could return them. With other values that weren’t critical to driving the car—like the fuel level, engine coolant temperature, etc.—we polled at a 1-2 second interval. For GPS, we subscribed to the LocationChanged event, which would fire when the GPS values changed.

Rather than creating a new serial port connection for every WCF request for OBD data, we created a singleton service that is instantiated when the service first runs. Accordingly, there is only one object in the WCF service that represents the last OBD and GPS data returned, which is obtained by the continual reading of the latest OBD data using the OBD library as described above. This means that calls to the WCF service ReadMeasurement method didn’t actually compute anything, but instead serialized the last saved data and returned it via the WCF service.

Since WCF supports multiple protocols, we implemented HTTP and TCP and ensured that any WCF service options we chose worked on Windows Phone, which, for example, can only use basic HTTP bindings.

To enable the ability to change the programming model later and to simplify the polling of the service, we built a helper library for Windows and Windows Phone that abstracts all the WCF calls.

The code below creates a new ObdService class and signs up for an event when the measurement has changed. The Start method does a couple of things: it lets you set the interval that you want to poll the ObdService, in this case every second (while the instrument cluster needs fast polling, the database logger can poll once a second). It also determines what IP address the service is hosted at (localhost), the protocol (HTTP or TCP), and whether to send “demo mode” data. Since one of the main ways the car is showcased is when it’s stopped on display, “demo mode” sends fake data, instead of always returning 0's for MPH, RPM, etc., so people can see what the instrument cluster would look like in action.

_service = new ObdService();
_service.ObdMeasurementChanged += service_ObdMeasurementChanged;
_service.Start(new TimeSpan(0, 0, 0, 0, 1000), localhost, Protocol.Http, false);

void service_ObdMeasurementChanged(object sender, ObdMeasurementChangedEventArgs e)
{
  Debug.Writeline("MPH=” + e.Measurement.MilesPerHour);
}

OBD-II Database & Azure Services

To record and capture the car telemetry data like MPH, RPM, engine load, and throttle (accelerator) position, as well as location data (latitude, longitude, altitude, and course), we used a SQL Server Express database with a simple, flat Entity Framework model, shown below. The primary key, the ObdMeasurementID is a GUID that is returned via the ObdService.  Just like above, the database logger subscribes to the ObdMeasurementChanged event and receives a new reading at the time interval set in the Start() method.

The Windows Azure data model uses Azure Table Services instead of SQL Server. The data mapping is essentially the same since both have a flat schema.

For Azure Table Storage, in addition to the schema above, you also need a partition key and a row key. For the partition key, we used a custom TripID (GUID) to represent a Trip. When the car is turned on/off a new TripID is created. That way we could group all measurements for that particular trip and do calculations based on that trip, like the average miles per gallon, distance traveled, fastest speed, etc. For the row key, we used a DateTimeOffset and a custom extension method, ToEndOfDays() that provides a unique numerical string (since Azure's row key is a string type) that subtracts the time from the DateTime.Max value. The result is that the earlier a DateTime value, the larger the number.

Example:

Time=5/11/2012 9:14:09 AM, EndOfDays=2520655479509478223 //larger
Time=5/11/2012 9:14:11 AM, EndOfDays=2520655479482804811 //smaller

Since they are ordered in reverse order, with the most recent date/time being the first row, we can write an efficient query to pull just the first row to get the current latitude/longitude without needing to scan the entire table for the last measurment.

public override string RowKey
{
    get
    {
        return new DateTimeOffset(TimeStamp).ToEndOfDays();
    }
    set
    {
        //do nothing
    }
}

public static class DateTimeExtensions
{
  public static string ToEndOfDays(this DateTimeOffset source)
  {  
    TimeSpan timeUntilTheEnd = DateTimeOffset.MaxValue.Subtract(source);
    return timeUntilTheEnd.Ticks.ToString();
  }

  public static DateTimeOffset FromEndOfDays(this String daysToEnd)
  {
    TimeSpan timeFromTheEnd = newTimeSpan(Int64.Parse(daysToEnd));
    DateTimeOffset source = DateTimeOffset.MaxValue.Date.Subtract(timeFromTheEnd);
    return source;
  }
}

To upload data to Azure, we used a timer-based background uploader that would check to see if there was an internet connection, and then filter and upload all of the local SQL Express rows that had not been submitted to Azure using the Submitted boolean database field. On the Azure side, we used an ASP.NET MVC controller to submit data. The controller deserializes the data into a List<MeasurementForTransfer> type, it adds the data to a blob, and adds the blob to a queue as shown below.

A worker role (or many) will then read items off the queue and the new OBD measurement rows are placed into Azure Table Storage.

public ActionResult PostData()
{
    try
    {
        StreamReader incomingData = new StreamReader(HttpContext.Request.InputStream);

        string data = incomingData.ReadToEnd();
        JavaScriptSerializer oSerializer =
            new JavaScriptSerializer();

        List<MeasurementForTransfer> measurements;
        measurements = oSerializer.Deserialize(data, typeof(List<MeasurementForTransfer>)) as List<MeasurementForTransfer>;

        if (measurements != null)
        {
            CloudBlob blob = _blob.UploadStringToIncoming(data);
            _queue.PushMessageToPostQueue(blob.Uri.ToString());
            return new HttpStatusCodeResult(200);
        }
        ...
    }
}

Instrument Cluster

Much of this is also covered in our previously released OBD-II library where the instrument cluster application is included as a sample. This is a WPF application that runs on a Windows 7 slate. It contains three different skins designed by 352 Media—a 2012 Mustang dashboard, a 1967 Mustang dashboard, and a Metro-style dashboard—each of which can be "swiped" through. This application queries the OBD-II WCF service described above as quickly as it can to retrieve speed, RPM, fuel level, and other data for display to the driver. The gauges are updated in real-time just as a real dashboard instrument cluster would behave.

HUD

The HUD (or Heads Up Display) application runs on one of the two Windows 7 computers in the car. This is a full-screen application that is output via a projector to a series of mirrors and a projection screen. This is then reflected onto the front glass of the windshield of the car. To install these, we altered the physical car's body and created brackets to mount mirrors and the projectors. In the picture on the left, you can see the dashboard's structural member pivoted outward. You can see the 12" section we removed and added in the base plate to allow light to be reflected through to the windshield. Bill Steele helped design and implement the physical HUD aspect into the car.

The HUD application has several different modes.  The mode is selected from the Windows Phone application.

• POI / Mapping – This uses Bing Maps services. The phone or Windows 8 passenger application can choose one of a select group of categories (Eat, Entertain, Shop, Gas). Once selected, the REST service layer is contacted and the current choice is persisted. The HUD is constantly polling the service to know what the current category is, and when it changes, the HUD switches to an overhead map display with the closest locations of that category displayed, along with your always updated current GPS position and direction. The list of closest items in the category is requested every few seconds from the Bing Maps API and the map is updated appropriately.
  
• Car telemetry - In the car telemetry mode, the OBD data from the WCF service described above is queried and displayed on the screen.  This can be though of as an overall car "status" display with the speed, RPMs, real-time MPG, time, and weather information.
• Weather – We use the World Weather Online API to get weather data for display on the HUD. This API allows queries for weather based on a latitude and longitude, which we have at all times. A quick call to the service gives us the current temperature and a general weather forecast, which we display as an icon next to the temperature in the lower-left portion of the screen.
  
• Kinect – Using our Kinect Service, with the standard WPF client code, we can display the rear camera on the HUD to help the driver when backing up. See the Kinect Service project for more information on how this works and to use the service in an application of your own.

Windows Phone Application

One of the main ways to control the vehicle is through the Windows Phone application. 

The first pivot of the app allows the user to lock, unlock, start the car, and set off the alarm.  This is done through the Viper product from Directed Electronics.

The second pivot contains the remaining ways that a user can interact with the car.

• Kinect – This uses the Kinect service much in the way the HUD does. It can display both the front and rear cameras as well as allow the user to listen to an audio clip and send it up to the car while applying a voice changing effect.
  
• Voice Effect – When the Talk button is pressed, the user can record their voice via the microphone. When released, the audio data is packaged in a simple WAV file and uploaded to the REST service. The user can select from several voice effects, such as Chipmunk and Deep. On the service side, that WAV file is modified with the selected effect and then played through the PA system. The code in this section of the app is very similar to the Coding4Fun Skype Voice Changer. We use NAudio and several pre-made effects to process the WAV file for play.
• Lighting – This controls the external lighting for the car. The user can select a zone, an animation, and a color to apply. Once selected, this is communicated through the REST service to the lighting controller.
  
• Messaging – This presents a list of known pictures and videos for the user. The selection is sent to the car through the REST service and displayed on the projector that is pointed at the rear window, allowing following drivers to see the image, video, or message. 
• Point Of Interest – As described earlier, this is the way the user can turn on the Point of Interest map on the HUD.  Selecting one of the four items sends the selection to the REST service where it is persisted. The polling HUD will know when the selection is changed and display the map interface as shown above.
• Telemetry – This is a replica of the instrument cluster that runs on the Windows 7 slate. OBD data is queried via the WCF service, just like the slate, and displayed on the gauges, just like the slate.
   
• Projection Screen – This will raise and lower the projection screen on the rear of the car.
• Horn – This displays a list of known horn sound effects that live on the REST service layer. Selecting any of the items will send a command through the REST to play that sound file on the external sound system of the car. This selected audio file would play when the horn was pressed in the car.
• Settings – Internal settings for setting up hardware and software for the car.

Passenger Application

The passenger interface runs on a Samsung Series 7 slate running the Windows 8 Consumer Preview. This interface has a subset of the functionality provided by the Windows Phone application, but communicates through the same REST service. From this interface, the passenger can set the car horn sound effect, view the front and back Kinect cameras, select a Point of Interest category to be displayed on the HUD, and select the image, video or message that will be displayed on the rear window.

External Car Lighting

The external lighting system was controlled by a web server running on a Netduino Plus using a Sparkfun protoshield board to simplify wiring, and allow for another shield to be used. The actual lights were Digital Addressable RGB LED w/ PWM. We'll also have a more in-depth article on this system on Coding4Fun shortly.

The car is broken down into different zones—grill, wheels, vents, etc. It also has a bunch of pre-defined procedural animation patterns that have a few adjustable parameters that allow for things like a snake effect, a sensor sweep, or even a police pattern. Each zone has its own thread which provides the ability to have multiple animation patterns going at the same time. When a command is received, the color, pattern, zone, and other data is then processed.

Here is a basic animation loop pattern.

private static void RandomAnimationWorker()
{
    var leds = GetLedsToIlluminate();
    var dataCopy = _data;
    var r = new Random();

    while (IsThreadSignaledToBeAlive(dataCopy.LightingZone))
    {
        for (var i = 0; i < leds.Length; i++)
            SetLed(leds[i], r.Next(255), r.Next(255), r.Next(255));

        LedRefresh();
        Thread.Sleep(dataCopy.TickDuration);
    }
}

Rear Projection Window

The rear projection system consists of two 4” linear actuators, a linear actual controller, the NETMF web server from above, a Seeed Studio Relay Shield, the back glass of a 1967 Ford Mustang, some rear projection film, a low profile yet insanely bright projector that accepts serial port commands, and a standard USB to serial adapter.

The REST service layer toggles the input of the projector based on the selected state.  This would allow us to go from the HDMI output of an Xbox 360 to the VGA output of the laptop.  While doing this, the REST layer sends a command to the NETMF web server to either raise or lower the actuators.

Here is the code for the NETMF to control the raising and lowering the glass:

public static class Relay
{
    // code for for Electronic Brick Relay Shield
    static readonly OutputPort RaisePort = new OutputPort(Pins.GPIO_PIN_D5, false);
    static readonly OutputPort LowerPort = new OutputPort(Pins.GPIO_PIN_D4, false);

    const int OpenCloseDelay = 1000;

    public static bool Raise()
    {
        return ExecuteRelay(RaisePort);
    }

    public static bool Lower()
    {
        return ExecuteRelay(LowerPort);
    }

    private static bool ExecuteRelay(OutputPort port)
    {
        port.Write(true);
        Thread.Sleep(OpenCloseDelay);
        port.Write(false);

        return true;
    }
}

Messaging System

This is a WPF application that leveraged the file system on the computer to communicate between the REST service layer and itself.  Visually, it shows the message/image/video in the rear view mirror but it actually does two other tasks, it operates our car horn system and plays the recorded audio output from the phone.

• Displaying Messages
  To display images, we poll the REST service every second for an update.  Depending on the return type, we either display a TextBlock element or a MediaElement.
  
  
• Detecting and Playing Car Horn
  When someone presses the horn in the car, it is detected by a Phidget 8/8/8 wired into a Digital Input. In-between the car horn and the Phidget, there is a relay as well. This isolates the voltage coming from the horn and solves a grounding issue. We then feed back two wires from that relay and put one into the ground and the other into one of the digital inputs. In the application, we listen to the InputChange event on the Phidget and play / stop the audio based on the state.
  
  
• Detecting new recorded audio from the phone and car horn changes
  When someone talks into the phone or selects a new car horn, the REST service layer places that audio file into a predetermined directory. The Messaging service then uses a FileSystemWatcher to detect when this file is added. The difference between the car horn detection and recorded audio is the recorded audio will play once it is done writing to the file system.

External PA System

To interact with people, we installed an external audio PA or Public Address system. This system is hooked into the laptop that is connected to the car horn, and can play audio data from the phone. Having a PA system that is as simple as an audio jack that plugs into a PC enabled us to have different ringtones for the car horn and to talk through the car using Windows Phone.

Conclusion

After months of planning and building, the Project Detroit car was shown to the world on an episode of Inside West Coast Customs. Though it was a ton of work, the end product is something we are all proud of. We hope that this project inspires other developers to think outside the box and realize what can be done with some off-the-shelf hardware, software, and passion.

OBD-II

OBD stands for On-Board Diagnostics.  In the world of cars, this can be thought of as the computer which has a variety of data points which can be queried, such as speed, fuel level, and even trouble codes that relate to the check engine light.  If you've ever taken your car in for service, your service center has hooked up the car to a computer via the OBD port to get the status of the car and what may not be working properly.

Getting Started

To use the OBD library with your vehicle, first you need an OBD to USB or serial cable.  These can be found in numerous places, but we recommend this cable from ScanTool.net, which was used with the Project Detroit car and our software.  Here's how to get connected:

1. With the car off, plug the OBD end into your vehicle.  The port is in a different location in every vehicle, but it's guaranteed to be somewhere near the driver's location.
2. Plug the USB/serial end of the cable into your computer and note which COM port the cable is connected to.
3. Update the app.config file of the InstrumentCluster project with that COM port.
4. Start the car.
5. Run the InstrumentCluster application, and data should start flowing.

Calling the OBD library programmatically

• Create an instance of the ObdDevice class
• Call the Connect method with the appropriate COM port, baud rate, and, if known, OBD protocol
• From here, the library works in two ways: automatic polling or manual polling
  • Automatic – Pass true to the Connect method. This will poll the OBD as quickly as it can requesting RPM, MPH, MPG, Fuel Level and Engine Coolant Temperature. Note that not all vehicles will support all of these. In this mode, hook the ObdChanged event to receive an ObdState packet with the new data.
  • Manual – Pass false to the Connect method. Then, create your own thread and call the ObdDevice's Get methods to retrieve the data you want as quickly as you want it.  This is the way we used the library in Project Detroit.  We polled manually, collecting the speed, RPM, and other critical data as quickly as possible, while only reading things like fuel level every few seconds.

OBD is a polled system, so data can only be returned as quickly as the time it takes to request it and return it. In short, only retrieve what you need when you need it.

ObdDevice obd;

obd = new ObdDevice();
obd.ObdChanged += _obd_ObdChanged;

// com port, baud rate, protocol (if known), automatic polling
obd.Connect("COM1", 115200, ObdDevice.UnknownProtocol, true);

...

void obd_ObdChanged(object sender, ObdChangedEventArgs e)
{
    e.ObdState.ToString();
}

Customizing the Instrument Cluster

The Instrument Cluster application is a WPF application that has 3 different skins which can be viewed by swiping left or right in the application, either with a finger, or the mouse.  This is a simplified version of the actual application we used in the Project Detroit car, modified to talk straight to the OBD port instead of our intermediate WCF service.  Please note that this isn't a robust, drop-in replacement application for a car dashboard, but only a sample of how the OBD library can be used.

Internally, the application uses the OBD library and hooks the ObdChanged event.  The ObdChanged event returns a minimal amount of important data.  When the event fires, the the SetInstrumentClusterValues method is run, which sets the OBD values to each gauge in whichever skin is currently visible.

private void SetInstrumentClusterValues(ObdState measurement)
{
    foreach (var item in _skins)
    {
        if(!item.IsVisible)
            continue;
        item.IsMalfunctionVisible = measurement.MilLightOn;
        item.IsLowFuelVisible = item.Fuel < 10;
        item.MPG = measurement.MilesPerGallon;
        item.MPH = measurement.MilesPerHour;
        item.RPM = measurement.Rpm;
        item.Fuel = measurement.FuelLevel;
        item.Temperature = measurement.EngineCoolantTemperature;
    }
}

More Information

• Project Detroit
• More information on OBD-II
• ELM323/327 Datasheet
• OBD-II to USB cable

Known Issues / Limitations

• This is known to work with cables that uses the ELM323/327 chipset. The specific cable that was used to develop this library is available from ScanTool.net, however other ELM323/327 OBD-II to USB cables should work.
• There are a variety of different OBD-II signal protocols. This library only supports a few of them. If your car's OBD protocol is not supported, please feel free to add it and send us the changes to integrate back into the library!
• If the vehicle has multiple ECUs, the ECU with the most PIDs is used. The other ECUs are ignored.
• Not every standard PID is supported.
• No custom manufacturer/model-specific PIDs are supported.