howtoloadtestwebapp

How To: Load Test a Web Application

J. D. Meier, Prashant Bansode, Carlos Farre

Applies To

* Web Applications

Summary

This how to shows you, how to Load test a Web Application. For Load testing a Web application, identify the performance critical key scenarios, identify the workload profile for distributing all the load among the key scenarios, identify metric that you want to collect to verify against for your performance objective, create test cases which will be used to simulate to load test, use tools to simulate the load according to the test cases and capture the metrics and finally analyze the metric data captured during the tests. Load test helps to identify the bottle necks and maximum operating capacity of the application.

* Objectives
* Overview
* Summary of Steps
* Step 1 – Identify Key Scenarios
* Step 2 – Identify Workload.
* Step 3 – Identify Metrics
* Step 4 – Create Test Cases.
* Step 5 – Simulate Load.
* Step 6 – Analyze the Results
* Resources

Objectives

* Learn how to Load test a web application

Overview

You use load testing to determine application behavior under normal and peak load conditions. You incrementally increase the load from normal to peak load to see how your application performs with varying load conditions. You continue to increase the load until you cross the threshold limit for your performance objectives. For example, you might continue to increase the load until the server CPU utilization reaches 75 percent, which is your specified threshold. The load testing process lets you identify application bottlenecks and the maximum operating capacity of the application.

Input

The following is useful inputs for Load testing a web application
* Performance critical use cases and user scenarios
* Workload Characteristics
* Performance metrics, performance objectives.

Output

The main items captured during Load testing are
* Updated test plans - Test cases, used for load testing.
* Various performance measures like throughput, response time, resource utilization etc.
* Potential bottlenecks that need to be analyzed in the white-box testing phase
* Peak operating capacity
* Behavior of the application block at various load levels

Steps

* Step 1. Identify Key Scenarios
* Step 2. Identify Workload
* Step 3. Identify Metrics
* Step 4. Create Test Cases
* Step 5. Simulate Load
* Step 6. Analyze the Results.

Step 1. Identify Key Scenarios

Start by identifying your application's key scenarios. Scenarios are anticipated user paths that generally incorporate multiple application activities. Key scenarios are those for which you have specific performance goals or those that have a significant performance impact. Here are the steps for identifying the key scenarios.

* Identify all the scenarios for a web application, for example in a commerce application supports following user scenarios.
* Browse Catalog
* Search a product
* Place an order
* Identify the activities involved in each of the scenarios, for example order placing scenario will include the following activities.:
* Log on to the application.
* Browse a product catalog.
* Search for a specific product.
* Add items to the shopping cart.
* Validate credit card details and place an order.
* Identify the scenarios which are most commonly executed or they are resource intensive, those will be the key scenario used for Load testing. For example in a commerce application Browsing a catalog is most commonly executed scenario where as placing an order will be resource intensive scenario as it access database.
* The information for most commonly executed scenarios, for an existing web application can be obtained from log files.
* Identify the pages involved in each of the scenarios
* Extract the user requests for each page from log files.
* Get the number of times a scenario is executed by using following formula
Number of times a scenario is executed = Total number of page requests / Number of page request per scenario
* Calculate the above for each scenario, comparison will show which of the scenarios are most commonly executed.
* The information for most commonly executed scenarios, for a new web application can be obtained from market research, historical data, market trends etc.
* The information for resource intensive scenario can be obtained from design docs or the actual code implementation. The primary resources are the following:
* CPU
* Memory
* Disk I/O
* Network I/O

Step 2. Identify Workload

A workload profile consists of an aggregate mix of users performing various operations. Identify the workload associated with each of the identified key scenarios. Following steps show how to identify the workload.
* Identify the distribution / ratio of the work - For each key scenario, identify the distribution / ratio of the work. The distribution is based on the number users executing the scenario. For an existing web application this information can be provided from IIS log as described step 1. For a new web site this information can be based on market research, historical data, market trends etc.
For the example, the distribution of work various key user scenarios could be similar to that shown in the following Table
Work Distribution

User Scenarios	% of Work distribution
Browse	50
Search	30
Place Order	20
Total	100

* Identify the user loads - Identify the maximum expected concurrent users for the web application. Using the work distribution for each scenario calculate the % user load per key scenario. For the example, the distribution of load for key scenario could be similar to that shown in the following Table.

Load Distribution

User Scenarios	% of users	Users
Browse	50	250
Search	30	150
Place Order	20	100
Total	100	500

* Identify the user loads at normal and peak hours - Identify the maximum expected concurrent users for the web application at normal and peak hours. Using the work distribution for each scenario calculate the % user load per key scenario at normal and peak hours.

Load Distribution at peak hours

User Scenarios	% of users	Users
Browse	50	500
Search	30	300
Place Order	20	200
Total	100	1000

Load Distribution at normal hours

User Scenarios	% of users	Users
Browse	50	250
Search	30	150
Place Order	20	100
Total	100	500

For more information about how to create a workload model for your application, see "How to - Model Workload for Web Application" at <<Add url>>

Step 3. Identify Metrics

Metrics provide information about how close your application is to your performance goals. In addition, they also help you identify problem areas and bottlenecks within your application. Identify the metrics that are relevant to your performance objectives and those that help you to identify bottlenecks..

Here is the process of identifying the metrics
* Define question related to your application performance which can be easily tested. For example, what is the checkout response time when placing an order? How many orders are placed in a minute? These questions have definite answers.
* Get answers for the questions identified, which defines the quality goal for the application. For example checkout response time should be 30 seconds and maximum 10 orders should be placed in a minute. The answers are based on marked research, historical data, market trends etc.
* With the list of performance related questions and answers, you set out to identify the metric. Inline with examples here is a sample of a high level metric to indicate, how your web application serves against the quality goals. Here is an example of a metric identified thus.

Metric	Performance Goal
Checkout Response time	30 seconds
Orders / minute	10

* Using the same approach you can identify lower level metrics which focus on measuring the performance and identifying the bottlenecks in the system. When identifying low level metric, you should determine a baseline for them under normal load conditions. This helps you decide on the acceptable load levels for your application. Baseline values help you analyze your application performance at varying load levels. Here is an example of low level metric.

Metric	Accepted level
Request Execution time	Must not exceed 8 seconds
Request Rejected	Less then 5
Throughput	100 or more requests / second
% process time	Must not exceed 75%
Memory Available	25 % of total RAM
Process Private Bytes	??

Additionally, to evaluate the performance of your application in more detail and to identify the potential bottlenecks, monitor metrics under the following categories:
* Network-specific metrics. This set of metrics provides information about the overall health and efficiency of your network, including routers, switches, and gateways.
* System-related metrics. This set of metrics helps you identify the resource utilization on your server. The resources are CPU, memory, disk I/O, and network I/O.
* Platform-specific metrics. Platform-specific metrics are related to software that is used to host your application, such as the .NET Framework common language runtime and ASP.NET-related metrics.
* Application-specific metrics. These include custom performance counters inserted in your application code to monitor application health and identify performance issues. You might use custom counters to determine the number of concurrent threads waiting to acquire a particular lock or the number of requests queued to make an outbound call to a Web service.
* Service level metrics. Service level metrics can help to measure overall application throughput and latency, or they might be tied to specific business scenarios.

Step 4. Create Test Cases

The test cases are created based on the scenarios and the profile mix identified in the previous steps. In general, the inputs for creating the test cases are performance objectives, workload characteristics, and the identified metrics. Each test case should mention the expected results in such a way that each test case can be marked as a pass or fail after execution.

For example for creating a test case for “Placing a Order” scenario of a commerce web application, following are the inputs required.

* Scenario – Placing a order
* Total number of concurrent users for the web application at normal load condition – 500 users
* # of users for the scenario – From load distribution for normal load the Placing an order scenario has 100 users
* Test duration – 2 hours
* Think time - Random think time between 1 and 10 seconds in the test script after each operation

Expected Results

The expected results for the commerce web application might be defined as the following, derived from the metrics identified in step3:
* Throughput: 100 requests per second (ASP.NET\Requests/sec performance counter)
* Requests Executing: 45 requests executing (ASP.NET\Requests Executing performance counter)
* Avg. Response Time: 8 second response time (TTLB on 100 megabits per second Mbps LAN)
* Resource utilization thresholds:
* Processor\% Processor Time: 75 percent
* Memory\Available MBytes: 25 percent of total physical RAM

Step 5. Simulate Load

Following are the steps involved in simulating the load
* Set up the environment with the minimum number of servers possible. Make sure that the architecture of your test setup mirrors your production environment as closely as possible.
* Use load generation tools to create web or unit tests inline with the identified test cases to be run for load testing.
* Using the load generation tools generate load for number of users specified in the workload characteristics for the given scenario.
* Begin load testing with small number of users distributed against your user profile, and then start to incrementally increase the load. It is important to have sufficient time between each step of increasing number of users, so that the system gets time to stabilize before next set of user connection executes the test case.
* Continue to increase the load, and record the behavior until you reach the threshold for the resources identified in your performance objectives.
* You can also continue to increase the number of users until you hit your service level limits, beyond which you would be violating your service level agreements for throughput, response time, and resource utilization.

Note – Make sure the client computers you use to generate load are not overly stressed. Resource utilization such as processor and memory should be well below the utilization threshold values.

Step 6. Analyze the Results

Following are the steps for analyzing the data.
* Analyze the captured data and compare the results against the metric's accepted level to determine whether the performance of the application being tested shows a trend toward or away from the performance objectives. For example for "Place a Order" scenario data captured for following measures is compared with the accepted level identified in Step 3 – Identify Metrics
* Request Execution time
* Throughput
* % process time
* Memory Available
* Process Private Bytes
* Analyze the measured metrics to diagnose potential bottlenecks. Based on the analysis, if required capture additional metrics in the subsequent test cycles. For example, suppose that during the first iteration of load tests, the process shows a marked increase in the memory consumption, indicating a possible memory leak. In the subsequent iterations, additional memory counters related to generations can be captured to study the memory allocation pattern for the application.

For a general template for creating the load testing report, see the "Reporting" section in "Chapter 16—Testing .NET Application Performance" of Improving .NET Application Performance and Scalability on MSDN at:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/scalenetchapt16.asp.

Resources

* For more information about tuning, see, "Chapter 17 — Tuning .NET Application Performance" At http://msdn2.microsoft.com/en-us/library/ms998583.aspx
* For more information about using ACT for performance testing, see "How To: Use ACT to Test Performance and Scalability" at http://msdn2.microsoft.com/en-us/library/ms979202.aspx
* For more information about using the Load Test tools available in Visual Studio 2005, Team Suite, see " How To: Create a Load Test Using VS 2005" at <<Add url>>.

Return to PerformanceTestingGuidance