Fall Fury: Part 4 - XAML Interop

As mentioned earlier in the series, FallFury does not solely rely on DirectX to display content to the user. As a Windows Store game, FallFury leverages the new Direct2D (XAML) project template, available in Visual Studio 2012.  Check out the video for this article at http://channel9.msdn.com/Series/FallFury/Part-4-XAML-Interop.  For a complete, offline version of this series, you may download a nicely formatted PDF of all the articles.

The Concept of a Swap Chain

Before I go into detail about the DirectX and XAML interop in FallFury, I want to cover one important aspect of DirectX development that you need to familiarize yourself with: the swap chain.

When your graphics adapter draws on the visual surface, you, as the user, see only minor potential redraws. Internally, however, the device switches buffers that reflect the displayed content, with each buffer representing a frame that has to be drawn. You can deduce from this that any swap chain has at least two buffers that it can switch between.

For example, if I want to display my character as being displaced by a specific amount of pixels, the buffer will at the outset present to me the character in its initial position, while the second buffer will be constructed in the background with the proper position adjustments. The first frame, made from the content from the first buffer, will be discarded, and then the second frame will be displayed, and so on. This process occurs at a very high speed that depends on the processing capabilities of the graphics adapter, so the user does not notice the swapping itself.

The most common swap chain is composed of two buffers—the screenbuffer and the secondary framebuffer.

SwapChainBackgroundPanel

DirectX interoperability with XAML simplifies a lot of routine tasks that would otherwise be handled with manual rendering procedures, such as a menu system or a simple game HUD. That being said, the way the XAML workflow is organized in a Direct2D project is quite different compared to a standard XAML Windows Store or Windows Phone application.

The core difference is that there is no navigational system per-se and the fundamental entity in a Direct2D (XAML) project that manages the XAML content is a SwapChainBackgroundPanel control. This control allows the developer to overlay XAML elements on top of the DirectX renders. It replaces the normal page-based layout with one in which it is sole container for every XAML control that has to be used in the application. This necessitates that you will have to organize the secondary elements in such a way that the correct set is displayed for the current application state.

For example, if the user is in the main menu, menu options as well as the game logo should be shown. When the user switches to the game mode, the HUD should appear and the menu should become hidden. Although both the menu and the HUD are a part of the same SwapChainBackgroundPanel, I will have to manually manage state and visibility changes.

Using the SwapChainBackgroundPanel also means that you will have to enforce specific graphic configuration rules in your application. One of them applies to setting up the swap chain. When you set up the scaling, it must be set to DXGI_SCALING_STRETCH:

DXGI_SWAP_CHAIN_DESC1 swapChainDesc = {0};
swapChainDesc.Width = static_cast<UINT>(m_renderTargetSize.Width); 
swapChainDesc.Height = static_cast<UINT>(m_renderTargetSize.Height);
swapChainDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
swapChainDesc.Stereo = false; 
swapChainDesc.SampleDesc.Count = 1;
swapChainDesc.SampleDesc.Quality = 0;
swapChainDesc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
swapChainDesc.BufferCount = 2;
swapChainDesc.Scaling = DXGI_SCALING_STRETCH;
swapChainDesc.SwapEffect = DXGI_SWAP_EFFECT_FLIP_SEQUENTIAL;
swapChainDesc.Flags = 0;

 

The swap chain itself should be configured for composition, mixing the native DirectX buffer with the overlaid XAML. This is done by calling CreateSwapChainForComposition:

ThrowIfFailed(
    dxgiFactory->CreateSwapChainForComposition(
        m_d3dDevice.Get(),
        &swapChainDesc,
        nullptr,
        &m_swapChain
        )
    );

ComPtr<ISwapChainBackgroundPanelNative> panelNative;
ThrowIfFailed(
    reinterpret_cast<IUnknown*>(m_panel)->QueryInterface(IID_PPV_ARGS(&panelNative))
    );

ThrowIfFailed(
    panelNative->SetSwapChain(m_swapChain.Get())
    );

There really isn’t much additional configuration work beyond this point. Remember, that the XAML content will always be overlaid on top of the DirectX content; therefore, plan the game components accordingly.

The XAML Menu System

The menu is at the core of the FallFury experience. When designing it, I was inspired by how Dance Central built the user interaction mechanism, and I tried to implement a similar approach in which the user would have to slide the button to the right instead of simply tapping on it:

clip_image002

Each menu item is built around a custom MenuItem control (MenuItem.xaml):

<UserControl
    x:Class="Coding4Fun.FallFury.MenuItem"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:Coding4Fun.FallFury"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    x:Name="menuItem"
    Margin="10, 0, 0, 10">

    <Grid 
        Height="65" 
        ManipulationMode="TranslateX"
        ManipulationCompleted="Grid_ManipulationCompleted"
        ManipulationDelta="Grid_ManipulationDelta" 
        x:Name="ControlContainer"
        PointerPressed="Grid_PointerPressed" 
        PointerReleased="Grid_PointerReleased">
        
        <Grid.Resources>
            <Storyboard x:Name="ArrowAnimator">
                <DoubleAnimation Storyboard.TargetName="ImageTranslateTransform"
                                 Storyboard.TargetProperty="X"
                                 From="0"
                                 To="20"
                                 Duration="0:0:0.4"
                                 RepeatBehavior="Forever"
                                 AutoReverse="True">
                </DoubleAnimation>
            </Storyboard>

        </Grid.Resources>
        
        <Grid.RowDefinitions>
            <RowDefinition Height="10" />
            <RowDefinition />
            <RowDefinition />
            <RowDefinition Height="10" />
        </Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="10" />
            <ColumnDefinition />
        </Grid.ColumnDefinitions>
        <StackPanel Orientation="Horizontal" Grid.RowSpan="4" Grid.ColumnSpan="4">
            <!-- width is sent in code behind, have to get this dynamic ...  -->
            <Grid Width="350" x:Name="coverRectangle">
                <Rectangle Fill="#303030" />
                <Rectangle 
                    Fill="Red" 
                    x:Name="coverActiveRectangle" />
            </Grid>
            <Image x:Name="MenuImage" Source="ms-appx:///MenuItems/single_arrow.png" Margin="10,0,0,0" Stretch="Uniform">
                <Image.RenderTransform>
                    <TranslateTransform x:Name="ImageTranslateTransform"></TranslateTransform>
                </Image.RenderTransform>
            </Image>
        </StackPanel>
        <!---->

        <TextBlock 
            Text="{Binding ElementName=menuItem, RelativeSource={RelativeSource Self}, Path=Label}"
            Grid.RowSpan="4"
            Grid.ColumnSpan="4"
            Style="{StaticResource MenuItemText}">

        </TextBlock>
        
        <MediaElement x:Name="coreMenuMedia" Source="ms-appx:///Assets/Sounds/MenuTap.wav" AutoPlay="False"></MediaElement>
        <MediaElement x:Name="slideMenuMedia" Source="ms-appx:///Assets/Sounds/MenuSlide.wav" AutoPlay="False"></MediaElement>
    </Grid>
</UserControl>

There is a core storyboard that is used to perform an image bounce. Initial tests showed that that most users tend to follow the standard “click-and-go” pattern and even though the implemented button was a slider, they tried to click it at least once, expecting the game to take them to the next screen. To avoid this, I introduced a visual arrow indicator that bounces left and right to indicate the direction of the necessary slide.

When the sliding occurs, the overlay grid width is being adjusted relative to the manipulation. At the same time, the button changes from passive state (dark background):

clip_image003

to active state (red background):

clip_image004

Internally, the double-arrow image replaces the single-arrow one and is translated across the X-axis to create a bouncing effect:

void MenuItem::Grid_PointerPressed(Platform::Object^ sender, Windows::UI::Xaml::Input::PointerRoutedEventArgs^ e)
{
    BitmapImage^ image = ref new BitmapImage(ref new Uri("ms-appx:///MenuItems/double_arrow.png"));
    MenuImage->Source = image;
    
    coverActiveRectangle->Visibility = Windows::UI::Xaml::Visibility::Visible;
    ((Storyboard^)ControlContainer->Resources->Lookup("ArrowAnimator"))->Begin();
    
    coreMenuMedia->Play();
}

As the user drags the finger on the button, the red overlay size should be adjusted accordingly. The size adjustment can be easily tracked in the ManipulationDelta event handler for the focused grid:

void MenuItem::Grid_ManipulationDelta(Platform::Object^ sender, Windows::UI::Xaml::Input::ManipulationDeltaRoutedEventArgs^ e)
{
    double diff = e->Cumulative.Translation.X;
    
    if (diff > maxDeltaSize)
    {
        diff = maxDeltaSize;
    }
    else if (diff < 0)
    {
        diff = 0;
    }
    
    coverRectangle->Width = initialBarWidth + diff;
}

When the user action on the button is completed, the ManipulationCompleted event handler is triggered on the same overlay grid. If the relative drag hits the critical threshold, the action linked to the button should be invoked:

void MenuItem::Grid_ManipulationCompleted(Platform::Object^ sender, Windows::UI::Xaml::Input::ManipulationCompletedRoutedEventArgs^ e)
{
    float diff = e->Cumulative.Translation.X;
    
    if (diff > maxDeltaSize - 50) // slight buffer
    {
        slideMenuMedia->Play();
        OnButtonSelected(this, Label);
    }
    
    ResetElements();
}

There are also local sound effects that are played when the slider-button is tapped and moved to the end. Both are handled by separate MediaElement controls that avoid an internal sound file switch, calling instead the Play method as necessary. The OnButtonSelected event handler can be dynamically hooked in the application backend.

Each button also has a visible content area that can display a text label. It can be set through a DependencyProperty:

DependencyProperty^ MenuItem::_LabelProperty = 
    DependencyProperty::Register("Label", 
    Platform::String::typeid,
    MenuItem::typeid, 
    nullptr);

In its current configuration, the menu label can be also set in XAML:

<local:MenuItem x:Name="btnNewGame" Label="new game">
    <local:MenuItem.RenderTransform>
        <TranslateTransform></TranslateTransform>
    </local:MenuItem.RenderTransform>
</local:MenuItem>

When the button loses the focus, the state resets to the passive one, stopping the animation and resetting the arrow image to the single one:

void MenuItem::ResetElements()
{
    BitmapImage^ image = ref new BitmapImage(ref new Uri("ms-appx:///MenuItems/single_arrow.png"));
    MenuImage->Source = image;
    
    ((Storyboard^)ControlContainer->Resources->Lookup("ArrowAnimator"))->Stop();
    
    coverRectangle->Width = initialBarWidth;
    coverActiveRectangle->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
}

That’s about as complex as the menu item control will get. The menu container itself can be any Grid or StackPanel control. The way the menu items are used across the game states, there is no need to have a separate unified container.

The Main XAML Container & State Changes

Going back to DirectXPage.xaml, there are several parts of the XAML layout that should be highlighted. First and foremost, the game curtains—the static parts of the screen that are being displayed instead of blacking out parts of the viewport that are not being used when the game runs in landscape mode. Because the game includes a rectangular frame instead of stretching the entire playable area to the size of the screen, there is an unknown amount of unallocated visual space on both the right and left sides of the frame itself:

<Grid HorizontalAlignment="Left" x:Name="containerA">
    <Rectangle Fill="#09bbe3" />

    <Rectangle Width="10" HorizontalAlignment="Left">
        <Rectangle.Fill>
            <LinearGradientBrush StartPoint="0,0" EndPoint="1,0">
                <GradientStop Color="#a000" Offset="0.0"></GradientStop>
                <GradientStop Color="#0000" Offset="1.0"></GradientStop>
            </LinearGradientBrush>
        </Rectangle.Fill>
    </Rectangle>
</Grid>

<Grid HorizontalAlignment="Right" x:Name="containerB">
    <Rectangle Fill="#09bbe3" />

    <Rectangle Width="10" HorizontalAlignment="Right">
        <Rectangle.Fill>
            <LinearGradientBrush StartPoint="1,0" EndPoint="0,0">
                <GradientStop Color="#a000" Offset="0.0"></GradientStop>
                <GradientStop Color="#0000" Offset="1.0"></GradientStop>
            </LinearGradientBrush>
        </Rectangle.Fill>
    </Rectangle>
</Grid>

Although these two grids have set alignments, the actual location on the screen will be set in code-behind because the size will also be re-calculated. Also, I need to make sure that the application is in the full display mode—if it is snapped, there is no need to display the curtains because the visible area will be reduced to an overlay grid.

The curtain resize and visibility are determined in the UpdateWindowSize method:

void DirectXPage::UpdateWindowSize()
{
    bool visibility = true;
    if (ApplicationView::Value == ApplicationViewState::Snapped)
            visibility = false;
    
    float margin = (m_renderer->m_renderTargetSize.Width - 768.0f) / 2.0f;
    if (margin < 2.0)
            visibility = false;
    
    if (visibility)
    {
        containerA->Width =  margin;
        containerA->HorizontalAlignment = Windows::UI::Xaml::HorizontalAlignment::Right;
        containerB->Width =  margin;
        containerB->HorizontalAlignment = Windows::UI::Xaml::HorizontalAlignment::Left;
        containerA->Visibility = Windows::UI::Xaml::Visibility::Visible;
        containerB->Visibility = Windows::UI::Xaml::Visibility::Visible;
    }
    else
    {
        containerA->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
        containerB->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
    }
}

The snippet above ensures that the curtains will only be displayed when there is extra, unused space in addition to the 768 pixels taken by the playable area.

Let’s take a look at state-specific content that is being displayed whenever the game switches states. Because of the nature of DirectX interaction, there is no way for me to hook to specific event handlers from the native loop. Therefore, I need to constantly check that the content displayed is associated with the current state.

This can be done with the help of the SwitchGameState method:

void DirectXPage::SwitchGameState()
{
    switch (m_renderer->CurrentGameState)
            {
            case GameState::GS_FULL_WIN:
                    {
                            grdCompleteWin->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_PLAYING:
                    {
                            Hud->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_MAIN_MENU:
                    {
                            stkMainMenu->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_GAME_OVER:
                    {
                            UpdateResults();
                            ResultPanel->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            grdGameOver->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_SUBMIT_SCORE:
                    {
                            grdSubmitScore->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            txtSubmitScoreView->Text = StaticDataHelper::Score.ToString();
                            break;
                    }
            case GameState::GS_TOP_SCORES:
                    {
                            grdTopScores->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_WIN:
                    {
                            UpdateResults();
                            grdWinner->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            ResultPanel->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_ABOUT_SCREEN:
                    {
                            grdAbout->Visibility =  Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_LEVEL_SELECT_SINGLE:
                    {
                            animationBeginTime = 0;
            
                            stkLevelSelector->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            break;
                    }
            case GameState::GS_HOW_TO:
                    {
                            grdHowTo->Visibility = Windows::UI::Xaml::Visibility::Visible;
                            Storyboard^ howToInitialBoard = (Storyboard^)grdMain->Resources->Lookup("StoryboardChainA");
                            howToInitialBoard->Begin();
                            break;
                    }
        default:
                    break;
            }
    
            Storyboard^ loc = (Storyboard^)grdMain->Resources->Lookup("FadingOut");
            loc->Begin();
}

One thing you’ve probably noticed about the snippet above is the fact that there is no indicator showing that controls are being hidden when the state changes. Just calling SwitchGameState in the Update loop would cause multiple controls to be displayed at once. However, there is also the HideEverything method that goes through the visual tree and sets the Visibility to Collapsed for everything:

void DirectXPage::HideEverything()
{
    for (uint i = 0; i < grdMain->Children->Size; i++)
    {
        grdMain->Children->GetAt(i)->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
    }
}

 

HUD Interaction

The Heads-Up Display (HUD) is used to alert the player about the current state of the game and the game character. In FallFury, it is used to display three indicators: how many buttons the character collected, how much time it took the character to get to the current level part, and current character health. It is also the container for the Pause button.

Here is the visual representation:

clip_image006

Here is the underlying XAML:

<Grid x:Name="Hud" VerticalAlignment="Top" Visibility="Collapsed">
    <Grid.RowDefinitions>
        <RowDefinition Height="80" />
        <RowDefinition Height="Auto" />
    </Grid.RowDefinitions>
    <Grid.ColumnDefinitions>
        <ColumnDefinition />
        <ColumnDefinition />
        <ColumnDefinition />
        <ColumnDefinition Width="80" />
    </Grid.ColumnDefinitions>

    <Rectangle Fill="Black" Grid.ColumnSpan="4" />

    <Button
        Grid.Column="0"
        x:Name="btnPause"
        Click="btnPause_Click"
        Style="{StaticResource PauseButton}">
        <Image Source="ms-appx:///Assets/HUD/pauseButton.png" Stretch="None" /
    </Button>

    <StackPanel Grid.Column="1" Orientation="Horizontal">
        <Image Source="ms-appx:///Assets/HUD/buttonHud.png" Stretch="None" />

        <TextBlock
            x:Name="txtButtons"
            Text="0"
            Style="{StaticResource hudResult}"/>
    </StackPanel>

    <StackPanel Grid.Column="2" Orientation="Horizontal">
        <Image Source="ms-appx:///Assets/HUD/clockHud.png" Stretch="None" />

        <TextBlock
            x:Name="txtTimer"
            Text="00:00"
            Style="{StaticResource hudResult}"/>
    </StackPanel>

    <Image
        Grid.Column="3"
        Source="ms-appx:///Assets/HUD/heartHud.png"
        Stretch="None" />

    <Grid
        Grid.Column="3"
        Grid.Row="1">
        <Rectangle Fill="Black" Opacity=".8"/>
        <controls:HealthBar x:Name="healthBar"></controls:HealthBar>
    </Grid>
</Grid>

As I mentioned before, the DirectX layer cannot directly interact with the XAML layer. Therefore, there needs to be an intermediary binding class. As the game progress changes, the UpdateHUD method is called, taking a reference to the current DirectX screen and reading the game data from the character and the StaticDataHelper class, which is the container for the time elapsed:

void DirectXPage::UpdateHud(GamePlayScreen^ playScreen)
{
    healthBar->Update(playScreen->GameBear->CurrentHealth, playScreen->GameBear->MaxHealth);
    
    txtButtons->Text = StaticDataHelper::ButtonsCollected.ToString();
    
    // Find a better built-in string formatting code
    if (m_renderer->CurrentGameState == GameState::GS_PLAYING && !(StaticDataHelper::IsPaused) && playScreen->IsLevelLoaded)
    {
        StaticDataHelper::SecondsTotal += m_timer->Delta;
        
        txtTimer->Text = StaticDataHelper::GetTimeString((int)StaticDataHelper::SecondsTotal);
    }
}

As seen above, part of the HUD is taken by a health indicator control—HealthBar. It is a simple composite element made out of two overlaying rectangles, one of which is resized as the bear health changes:

<UserControl
    x:Class="Coding4Fun.FallFury.Controls.HealthBar"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:FallFury"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    d:DesignHeight="410"
    d:DesignWidth="20" Height="410" Width="20">
    
    <Grid>
        <StackPanel VerticalAlignment="Top" Margin="0, 5, 0,20">
            <Rectangle Height="400" Style="{StaticResource HealthRectangle}" />
        </StackPanel>

        <StackPanel VerticalAlignment="Top" Margin="0, 5, 0, 10">
            <Rectangle x:Name="OverlayStacker" Height="0" Style="{StaticResource HealthOverlayRectangle}" />
        </StackPanel>
    </Grid>
</UserControl>

Being of a fixed size, it is fairly easy to calculate the health-to-damage ratio and display that as the size of the overlaid rectangle:

DependencyProperty^ HealthBar::_MaxHealthProperty = 
    DependencyProperty::Register("MaxHealth", 
    double::typeid,
    HealthBar::typeid, 
    nullptr);

DependencyProperty^ HealthBar::_CurrentHealthProperty = 
    DependencyProperty::Register("CurrentHealth", 
    double::typeid,
    HealthBar::typeid, 
    nullptr);

void HealthBar::Update(double currentHealth, double maxHealth)
{
    CurrentHealth = currentHealth;
    MaxHealth = maxHealth;
    
    if (CurrentHealth >= 0)
    {
        OverlayStacker->Height = 400.0 - ((400.0 * CurrentHealth) / MaxHealth);
    }
}

Conclusion

Using XAML as a part of a DirectX application does not require the developer to do a massive overhaul of the infrastructure. That said, be mindful when deciding whether to use a hybrid XAML application, as it is much easier to integrate a SwapChainBackgroundPanel with the underlying DirectX configuration from the ground up instead of trying to do so when the DirectX component is completed.

Tags:

Follow the Discussion

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.