Part 23: Testing and Submitting to the Store
Now that we've finished the app, it's time to test it and submit it to the Store.
A word of caution before we continue ... at a minimum, your users expect your app to be solid and to look professional.
With regards to creating a solid user experience, there's only so much you can do by yourself because you have the curse of knowledge. As the developer, your knowledge of the flow and "normal operation" of the app creates a myopia. You'll quickly find that your users will brake the app in new and creative ways. So, I would spend a of time thinking about the states of the application, particularly any scenario that accepts information from the user in input controls or dialogs. I would spend time thinking about exception handling and messages back to the user to help diagnose the problem. I would think about inputs and outputs and edge case scenarios ... What if they use strange characters outside of the ASCII or Extended ASCII range? What if they add two sounds with the same name? What if the program runs out of battery while the user is recording a sound? What happens if the user clicks one of the hardware buttons while recording or playing back a sound? What if the user fills up Isolated Storage with custom sounds? As it stands now, that would be a problem because we don't have a delete sound feature in the app.
With regards to a professional appearance, here again you have the curse of knowledge and quite possibly a cognitive bias that leads you to think the flow and aesthetics of the app look great when your users think otherwise. Getting feedback, especially from someone who may study design, usability, etc. could prove invaluable. I would get them involved early and often throughout the development process. The beauty of the Windows Design language, formerly known as Metro, is that it pushes you towards a certain aesthetic sensibility. You should study the recommendations from Microsoft in this regard:
At any rate, even when you're getting creative and thinking through the functionality of your app, you'll miss something. That's why it's good to:
- Let others see your work, especially those who are not your close friends or family. You want honest feedback. Your family and friends won't want to hurt your feelings if they don't like your work.
- Use the Beta test feature of the store before you attempt to sell the app. Now you're opening your app up to more scrutiny and feedback, however it's better you hear that feedback in beta rather than hear that feedback in the form of negative reviews that pour in and kill your app's chances at success.
- Even after you've sold the app (or distribute it for free), be responsive to customer emails, questions, feedback, etc. Whether you realize it or not, once you publish an app you're now running a business. From now on you need to be the friendliest guy on the planet. If someone asks for a new feature, ask them for more details and genuinely give serious thought to whether that's a viable feature to add or a bug to fix. If someone is frustrated and uses harsh language or tone against your app, you gain absolutely nothing by returning the salvo. You need to grow thick skin, swallow your pride, and be the bigger person. From personal experience in handling customers who are displeased over the years, you can turn an enemy into a friend more often by not by choosing your words carefully. May I recommend the time-honored book, How to Win Friends and Influence People by Dale Carnegie.
- Dedicate a URL and a few pages to your app. Include FAQ's so people can self-serve for the most common issues.
- Understand the impact of low-memory on your app. I suspect most app crashes have to do with memory constraints, or the failure to close a resource like the file system when you're finished working with them.
The game plan for this lesson:
- We'll perform some compliance testing to make sure our app has a chance to be included in the Store
- We'll submit our app to the store
1. Run the Store Test Kit
To evaluate the readiness of our app for the app store, we'll use the Store Test Kit. This will automatically test a few scenarios and suggest other manual tests for you to conduct. Furthermore, it works as a check list for the items you'll need to submit to the store, like promotional images and the like.
To open the Store Test Kit, go to the Project menu select Open Store Test Kit:
This will open a new page in the main area. Notice there are three tabs on the left:
The Application Details collects basic data about the app, including a larger version of the main tile (300 pixels by 300 pixels) for the Store and screenshots of the app. If you use the Phone Store, you can see that each app has a page with descriptive text and details about the publisher, a page with the ratings and reviews for the app, and a page of screenshots.
The Automated Tests tab will perform a few tests to ensure the validity of the XAP deployment package, and will ensure the images (from the Application Details tab) are in place as well.
The Manual Tests tab has a number of suggestions for you to test on your app before submitting it.
To the right of the test status (think of this like a check box that you will manually check off) and the name of the test is the description of the test and how to perform the test, what constitutes failure, etc.
If you attempt to run the tests without changing anything in Visual Studio by clicking the "Run Tests" button on the Automated Tests tab ...
... you'll probably get the following warning / error:
There are several issues we'll need to correct, but the first one is that we need to create a Release version of the deployment package. We've been using a Debug version which includes additional files used to sync up the execution of the code on the Emulator or the Device with Visual Studio so that you can set break points, step through code, and so on.
Now we'll need to create a release version. Before I do that, let's confirm that Visual Studio will create a Release version that is configured to be deployed to "AnyCPU" per the instructions of this dialog. In the toolbar, I'll select the drop down list next to the Run button and select Configuration Manager:
The Configuration Manager dialog appears:
In the Action solution configuration drop down list, I'll select Release ...
And I'll review the Project contexts in the main area, ensuring that the Configuration is set to Release and the Platform is set to "Any CPU".
If the settings are right (and they should be by default), then I'll Build the Solution (Build menu, select Build Solution).
If all goes well, the Output window should indicate success. If it doesn't you'll obviously need to fix that before continuing.
Now that we have a valid release version of our XAP deployment package, we'll "Run Tests" again ...
... this time, we see that we failed with Iconography and Screenshots.
Before I set those, let's take a look at the button further down on that page ... the "Start Windows Phone Application Analysis" ...
Clicking this button will add a new .sap file which is a blank performance log file that will capture data about the performance of your app as it runs. The log file will then be poured over and reported on to let you know how well it runs in certain situations:
That .sap file is loaded into the main area of Visual Studio:
Rather than spend time talking about the different types of data and reports you can generate from the monitoring and profiling options, I would recommend that you read what the help has to say about it:
At this point, I'll merely:
- Choose the App Analysis monitoring option, then ...
- Click the Start Session link at the bottom.
The app will load in the Emulator and you'll see Visual Studio display a monitoring message:
At this point you should put the app through its paces and perform your typical use cases, and perhaps even some edge cases. When you're finished testing the app ...
- Click the "End Session" link
The collected data will be analyzed:
... and a report will be generated ...
The report includes some very interesting details about the app that could help you determine its readiness for the Store. I think one of the most important things to look at is the battery consumption and the max memory used.
Now that we've completed this test, I'll make sure to ready the images needed for the Store, such as the Store Tile (300x300) and the screenshots. I use the Emulator's Additional Tools to capture two main views of the SoundBoard app:
And I've chosen to save them into the Assets folder of the project. However, in retrospect, I probably would have put them somewhere else ... perhaps another subfolder of the project so I could get back to them more easily and have less confusion about the names and such.
I add the two screenshots to the Application Screenshots section of the Store Test Kit:
However, if you were to run the tests again, it will still fail because you need to provide versions for ALL THREE RESOLUTIONS.
In the Application Screenshots section of the Application Details page, use the drop down to reveal the other screen resolutions and populate them with screen shots using the different versions of the Emulator to capture screen shots at those resolutions.
TIP: When I went to upload the images to the Store, it provided an option to create / scale the various versions of these files based on the WXGA version. However, if getting the Store Test Kit to pass successfully is important to you, then you'll need images for each of the three resolutions.
After supplying all of the required files, I was able to get the automated tests to pass:
Again, I still have manual tests to consider, as well as my own internal testing I want to perform, but at least this helped me get the assets and the release version of the app ready for submission.
2. Submitting the app to the Store
Now I'll attempt to publish my app. This is a multi-day process. I'm not 100% confident that the app will pass the first time. However, I do want to give you an idea of the submission process. It's not scary at all.
It's also important to keep in mind that this site changes often, and it would be impossible to keep up with the changes. So, what you see when you attempt to do this may be dramatically different. You should always read the on-page instructions for yourself. The purpose of this lesson is to merely show you the general process up front before you actually build your app.
Go to: https://dev.windowsphone.com
- Click the Publish menu option at the top of the page,
- Click the Submit App link on the page
That will take you to the Submit app page:
This page contains a series of tasks, many of which are optional depending on the features of your app and how you hope to monetize it.
- Click on the App info task
That will take you to the App info page ...
Here you'll name, categorize, price and choose who you want to distribute the app to. Most of this is self explanatory. There's a lot to consider here, and honestly your choices here reflect your strategy ... who you will target the app to? What is your pricing strategy? If you are going to charge for your app (as opposed to a freemium or advertising model) then will you allow free trials? In some cases, this would make sense. In other cases, it might make less sense. I hear that, in general, trials do help sales but I'm sure that depends on the type of app and your target audience.
Further down on that page ...
You can choose which markets to distribute to, and whether you want to start in the Store or in a beta as we described earlier.
Even further down that page ...
You can choose whether to put it in the store automatically after it has been certified, or whether to
The final option is for MPNS, or rather, the Microsoft Push Notification Service for Windows Phone. This allows you to send messages from a server to a user's phone securely. Imaging a long running massively multi-player casual online game ... when the player's home planet is about to be attacked by another player, the game could notify that an attack is eminent. This might be a premium upgrade in the game. It would give the player a chance to move forces into place. This service costs money, so you need to use it judiciously. For more information, check this out:
Push notifications for Windows Phone
After you click the Save button (above) you'll return to the Submit app page with the first task marked as complete:
- Click task 2 to upload and describe the XAP file
That will take you to the Upload and describe your XAP page ...
- Click the Add new link ...
This will display the Open dialog:
- Navigate to the RELEASE version of the XAP in the project's Bin\Release directory.
- Select the version with the suffix: _Release_AnyCPU.xap
- Click Open
It may take a few moments to upload your XAP, especially if you have a lot of image or sound resources embedded in the XAP ...
Once the XAP is uploaded, you'll see that it is analyzed and its details are displayed:
Further down that page, you can modify the XAP's Store listing info ...
- For each language you support, you'll add ...
- A description for the app, and ...
- keywords your potential users might search for to find your app
I would recommend that you give careful thought to your descriptive text (as well as your screenshots). The effectiveness of your writing will either convince someone to give your app a try or move on to another app.
Even further down on that page you'll be able to upload images for the Store ...
To add an image, you click on the plus symbol in the middle of a given gray square. This will display the Open dialog where you can choose your file.
- As you can see, I've already selected the 300 x 300 pixel store tile.
- You can add a background 1000 x 800 that will be displayed for your app. This helps to brand your app and create excitement about the app.
- I mentioned this earlier ... you can either supply screen shots at each of the screen resolutions you'll support, or you can choose to allow the Store to automatically generate / scale versions of your images from the WXGA version. I chose that in this example.
- Here I uploaded the WXGA version of the screen shot images.
Even further down the page ...
... I click the Save button. This will return me to the Submit app page with the second task checked as complete ...
Further down that page, you see a number of optional tasks. In our case, we're not using Microsoft's in-app advertising platform, nor will we worry about customizing the pricing on a per-market basis, nor will we need to enable Map service which will provide some API credentials for use in our app. This last option, however, will be important for the next app we'll build. Just put a peg in that idea and we'll revisit it later in this series.
With the two required tasks complete, I click the "Review and Submit" button.
This will display the Review Submission page ...
The Review Submission page is your last chance to catch an error before you submit your app to the Store for review.
Nearer to the bottom of that page, you can see the option to Submit or "Go Back and Edit". I'll click the Submit button ...
... and this will give me the message that the App submission was successful ...
As the text says, it takes up to 5 business days and you'll be notified by email whether your app is certified for inclusion in the store or not.
Now we wait.
To recap, the main take away of this lesson was the thought process of testing, submitting, promoting and supporting your app. There are some tools like the Store Test Kit that can help you get your assets collected and ready for submission, but beyond that you'll want to get your app into the hands of others to try it and provide feedback throughout the entire development process, as well as create a beta program. These steps, along with good artwork, screenshots, descriptions and responsiveness to users after the sale help ensure the success of your app in the Store.