howtomanageperfcyclecmmi

How To: Manage the Performance Test Cycle (CMMI)

Applies to

* Performance Testing in an CMMI or Auditable Development Environment

Summary

This How To describes an industry proven approach to managing performance testing an application that is being developed using a CMMI, Auditable or Regulatory Compliant Software Development Environment. Managing Performance testing in a highly planned, monitored environment is particularly challenging because it is frequently impossible to conduct the next planned activity until any defects detected during the previous activity are completely resolved. The approach to performance test management presented in this How To will help you and your team deal with this challenge.

* Objectives
* Overview
* Definitions
* Step1: Understand Auditability/Compliance Criteria
* Step2: Collect Performance Requirements, Goals and Objectives
* Step3: Understand the System Being Developed and the Project Plan
* Step4: Draft a Performance Test Plan by Mapping Performance Requirements, Goals and Objectives to both the Project Plan and Auditabiliy/Compliance Criteria into Quantifiable Performance Test Work Items
* Step5: Flesh-out the Approved Draft, Including Alternate and Exception Case Work Items for each Planned Work Item
* Step6: Execute Planned, Alternate or Exception Case Work Items
* Step7: Report Results and Archive Performance Test Information, Scripts, etc.
* Step8: Modify Plan and Gain Approval for Modification (Triage)
* Step9: Return to Step6
* Additional Resources

Objectives

Blah

Overview

CMMI is used here as a paradigmatic example of a process that is generally viewed as being opposed to agility. CMMI is generally heavy-weight and typically appropriate for safety critical software and software that is subject to regulatory standards and/or process audits. For more about CMMI and the Microsoft Solutions Framework (MSF) for CMMI projects, see {reference}
The nature of performance testing makes it difficult to predict in advance when what type of test will add value or even be possible. Obviously, this leads to a planning challenge. In this HowTo, an industry validated approach to planning and managing performance testing is described. This approach is sensitive to the need for auditability, progress tracking and plan changes requiring approval without being oppressively procedural. The key to the approach is to plan at the Performance Test Work Item level and to fit those work items into the existing project plan. This allows for compliance, auditablity and approval gates while leaving the execution details in the hands of those assigned to complete a particular work item.

Definitions

The following definitions are used both within this Explain and throughout the Performance Testing Guidance. These terms also appear in the Glossary {link}, but are repeated here for easy reference. Every attempt has been made to ensure that these terms and definitions are consistent with formal and industrial use; however, some of these terms are known to have certain valid alternate definitions and implications in specific industries and organizations. The use of these definitions are intended to aid communication and are not an attempt to create a universal standard.
* Test Strategy: The test strategy is the way tests will be designed and executed to support an effective quality assessment.
* Test Logistics: The set of ideas that guide how resources are applied to fulfill the test strategy.
* Test Design: The process of creating tests.
* Test Execution: The process of configuring, operating, and observing a product for the purpose of evaluating it.
* Test Plan: The set of ideas that guide or represent the intended test process.
* Test Plan Document: Any document intended to convey test plan information. However, test plan documents are not the only source of information about the test plan. Test plan information is also contained in the oral tradition of the project and the culture of the company.
* Test Work Item: A test related activity that can be expressed as a unit of work that can be used to track progress.
* Planned Test Work Item: A test work item that is scheduled for completion at a particular time, or in a particular sequence.
* Alternate Test Work Item: A test work item that can substitute for a planned test work item if the planned test work item is not achievable at the designated time or sequence.
* Exception Test Work Item: A test work item that can substitute for a planned test work item if neither the planned nor alternate test work item is achievable at the designated time or sequence. Exception test work items should have as few dependencies as possible to ensure that testers do not get entirely stalled if the project or application ends up in an unanticipated state.

Step1: Understand Auditability/Compliance Criteria

Interview, read and research.

Step2: Collect Performance Requirements, Goals and Objectives

Reference other HowTo’s

Step3: Understand the System Being Developed and the Project Plan

Reference other HowTo plus discuss what to look for in the project plan

Step4: Draft a Performance Test Plan by Mapping Performance Requirements, Goals and Objectives to both the Project Plan and Auditabiliy/Compliance Criteria into Quantifiable Performance Test Work Items

The mapping is the key. Basically, define a Performance Test Work Item to satisfy each goal, requirement, objective and auditability/compliance issue, then plug those work items into appropriate places in the project plan.

Step5: Flesh-out the Approved Draft, Including Alternate and Exception Case Work Items for each Planned Work Item

Add detail to work items (resources required, etc). Ensure that every work item has an alternate and exception… ensure that every work item appears at least once as the planned item and at least once as either an alternate or exception.

Step6: Execute Planned, Alternate or Exception Case Work Items

Duh. 

Step7: Report Results and Archive Performance Test Information, Scripts, etc.

Refer to other HowTos for reporting, talking about archiving for auditability

Step8: Modify Plan and Gain Approval for Modification (Triage)

Generally, cross off the done stuff and shuffle the undone stuff into the best available slot.

Step9: Return to Step6

Season to taste.

Attributes

• Technology:
• Topic:
• Type: HowTo
• Category:
• Priority:
• Status:
• Source: VSTS Load Test Guide
• Author: Scott Barber