performancetestingoutliersmistake

Handle Outliers in Performance Test Reports

Content Type: How-To:
Author: Scott
Contributer:
Reviewer:

Summary:

Baselining the application is where test execution actually begins. The intent is twofold. First, all scripts need to be executed, validated and debugged (if necessary). Second, various single and multi-user tests are executed and recorded to provide a basis of comparison for all future testing. Initial baselines are typically taken as soon as the test environment is available. Re-baselining occurs at each new release.

Sravan Tadakamalla

Objectives:

* Learn how to design and execute single user baseline scenarios
* Learn how to design and execute multi-user baseline scenarios
* Learn how to report test results against baseline scenarios
* Learn how to determine when new baselines need to be taken

Notes:

Return to Table Of Contents

Sravan Tadikis

Handling outlier is an observation that is numerically distant from the rest of the data. Statistics derived from data sets that include outliers may be misleading in terms of response times or 90% tile. Example: - Generally if we have any failed transactions due to time outs errors, we need to find it out how this time out occurred in our execution. Is it because of processor queue length? Or running out memory in web server or application server? Especially we need to display graphs in our reports for these of issues in the environment. Some time due to network outage we may see some spike in while executing the tests.