Terms and Concepts

This page explains the main terms and concepts used throughout the guidance.

Key Terms

* Application Scalability Characteristics of the product under test related to the number of users the product can support. These Characteristics or Qualities of Service may be related to user load, network or data capacity and/or product failure modes related to the product's inability to scale beyond a particular level. - Scott Barber
* Application Speed Characteristics of the product under test related to the product's overall speed of response, or sub-system's speed of response, to a user initiated activity - Scott Barber
* Application Stability Characteristics of the product under test related to the product's overall reliability, robustness, functional and data integrity, availability and/or consistency of responsiveness under a variety of expected and unexpected conditions. - Scott Barber
* Application Usage Profile One or more descriptions of how the product under test is, or is anticipated to be, used during production operations. Usage profiles are typically expressed in terms of business activities and usage scenarios. - Scott Barber
* Bug Anything that threatens the value of the product. Something that bugs someone whose opinion matters. - James Bach
* Component Performance Test Any performance test that targets an architectural component of the application. Commonly tested components include servers, databases, networks, firewalls and storage devices. - Scott Barber
* Endurance Test A performance test focused on determining or validating performance characteristics of the product under test when subjected to workload models and load volumes anticipated during production operations over an extended period of time. Endurance testing is a subset of load testing. - Scott Barber
* Exploratory Testing An interactive process of simultaneous learning, test design, and test execution. - James Bach
* Heuristic Testing An approach to test design that employs heuristics to enable rapid development of test cases. - James Bach
* Latency Server latency is the time the server takes to complete the execution of a request. Server latency does not include network latency. Network latency is the additional time that it takes for a request and a response to cross a network. Client latency is the time that it takes for a request to reach a server and for the response to travel back.
* Load Test A performance test focused on determining or validating performance characteristics of the product under test when subjected to workload models and load volumes anticipated during production operations. - Scott Barber
* Metrics Metrics are the actual measurements obtained by running performance tests. These performance tests include system-related metrics such as CPU, memory, disk I/O, network I/O, and resource utilization levels. The performance tests also include application-specific metrics such as performance counters and timing data.
* Performance budgets Performance budgets are your constraints. Performance budgets specify the amount of resources that you can use for specific scenarios and operations and still be successful.
* Performance Performance is concerned with achieving response times, throughput, and resource utilization levels that meet your performance objectives.
* Performance objectives Performance objectives are usually specified in terms of response times, throughput (transactions per second), and resource utilization levels. Resource utilization levels include the amount of CPU capacity, memory, disk I/O, and network I/O that your application consumes.
* Performance Targets The desired value for a resource of interest under a particular set of conditions, usually specified in terms of response times, throughput (transactions per second), and resource utilization levels. Resource utilization levels include the amount of CPU capacity, memory, disk I/O, and network I/O that your application consumes. - Scott Barber
* Performance Testing Objective Information to be collected through the process of performance testing that is anticipated to have value in determining or improving the quality of the product, but are not necessarily quantitative or directly related to a performance requirement, goal or stated Quality of Service. - Scott Barber
* Performance Thresholds The maximum acceptable value for a resource of interest, usually specified in terms of response times, throughput (transactions per second), and resource utilization levels. Resource utilization levels include the amount of CPU capacity, memory, disk I/O, and network I/O that your application consumes. - Scott Barber
* Performance Unit Test Any performance test that targets a module of code where that module is any logical sub-set of the entire existing code base of the application. Commonly tested modules include functions, procedures, routines, objects, methods and classes. Performance Unit Tests are frequently created and conducted by the developer who wrote the module of code being tested. - Scott Barber
* Quality Something of value to some person. - Jerry Weinberg
* Response time. Response time is the amount of time that it takes for a server to respond to a request.
* Resource utilization Resource utilization is the measure of how much server and network resources are consumed by your application. Resources include CPU, memory, disk I/O, and network I/O. Resource utilization is the cost in terms of system resources. The primary resources are CPU, memory, disk I/O, and network I/O.
* Risk-Based Testing Any testing organized to explore specific product risks. - James Bach
* Scalability Scalability refers to the ability to handle additional workload, without adversely affecting performance, by adding resources such as CPU, memory, and storage capacity.
* Scenarios Scenarios are a sequence of steps in your application. They can represent a use case or a business function such as searching a product catalog, adding an item to a shopping cart, or placing an order.
* Software Performance Goals Performance related characteristics of the product under test that are desired to be met prior to product release, but which are not strictly mandatory. - Scott Barber
* Software Performance Investigation An activity based on collecting information related to speed, scalability and/or stability characteristics about the product under test that may have value in determining or improving the quality of the product. - Scott Barber
* Software Performance Requirements Performance related characteristics of the product under test that must be met in order for the product to be released. Performance requirements are mandated via legal contract or service level agreement. - Scott Barber
* Software Performance Testing A technical investigation done to determine or validate speed, scalability and/or stability characteristics of the product under test. - Scott Barber
* Software Performance Validation An activity that compares speed, scalability and/or stability characteristics of the product under test to the expectations that have been set or presumed for that product. - Scott Barber
* Software Testing A technical investigation done to expose quality-related information about the product under test. - Cem Kaner
* Spike Test A performance test focused on determining or validating performance characteristics of the product under test when subjected to workload models and load volumes that repeatedly increase beyond anticipated production operations for short periods of time. Spike testing is a subset of stress testing. - Scott Barber
* Stress Test A performance test focused on determining or validating performance characteristics of the product under test when subjected to workload models, and load volumes beyond those anticipated during production operations. Stress tests may also include tests focused on determining or validating performance characteristics of the product under test when subjected to workload models and load volumes when the product is subjected to other stressful conditions, such as limited memory, insufficient disk space or server failure. - Scott Barber
* Test Design The process of creating tests. - James Bach
* Test Execution The process of configuring, operating, and observing a product for the purpose of evaluating it. - James Bach
* Test Logistics The set of ideas that guide how resources are applied to fulfill the test strategy. - James Bach
* Test Plan The set of ideas that guide or represent the intended test process. - James Bach
* Test Strategy The way tests will be designed and executed to support an effective quality assessment.. - James Bach
* Throughput Throughput is the number of requests that can be served by your application per unit time. Throughput varies depending on the load. Throughput is frequently measured as requests or logical transactions per second.
* Workload Workload includes the total number of users and concurrent active users, data volumes, and transaction volumes. Workload is typically derived from marketing data. The workload includes total numbers of users, concurrent active users, data volumes, and transaction volumes, along with the transaction mix. For performance modeling, you associate a workload with an individual scenario.
* User Community Model Models that enhance the application usage profile(s) by adding distribution of activities, hourly usage volume and other necessary variables to design realistic performance tests. - Scott Barber

---

Return to PerformanceTestingGuidance