performancetestingtableofcontents

Table of Contents - Performance Testing Guidance

This page shows the current working version of the Table of Contents.

Introduction to Performance Testing

a) Introduce Key Terms and Concepts

* Explain or Principle: Types of Performance Testing -Scott
* Explain or Principle: Risks Addressed via Performance Testing -Scott
* Explain or Principle: Understanding Volume -Scott
* Explain or Principle: Test tools, test artifacts

b) Performance Test Strategy

* How-To: Evaluate Systems to Improve Performance Testing Effectiveness -Scott
* How-To: Determine Performance Testing Objectives, Building the Targets -Scott
* How-To: Quantify End Users Response Time Goals -Scott
* Test assets
* How-To: Baseline Application Performance -Scott
* Test Resources tools and personas. Who is the Performance tester?
* How-To: Manage the Performance Test cycle. (Agile) -Scott
* How-To: Manage the Performance Test cycle. (CMMI) -Scott
* How-To: Conduct a Performance Testing Risk Assessment -Scott

c) Environment - Performance Test Beds

* Creating
* Troubleshooting
* Real world simulation
* Server configurations
* Client Configurations
* Scalable test environments

d) Key Variables and Metrics

* How-To: Workload Characterization
* How-To: Client variables and metrics (Location, network connection, payloads)
* How-To: Server variables and metrics (Interaction speed, latency, software hardware configurations)
* How-To: Load Patterns in Web Applications
* How-To: Server Performance Metrics: data collection and monitoring

e) Test Execution

* How-To: VSTS solution
* How-To: Test Scripts (creating troubleshooting)
* How-To: Reporting and Analysis
* How-To: Data Presentations

Defining the Performance and Stress Testing

a) Modeling the User experience

* How-To: Simulate Realistic User Delays -Scott
* How-To: Model Representative User Groups -Scott
* How-To: Simulate Realistic User Patterns -Scott
* How-To: Account for User Abandonment -Scott
* How-To: Define User Sessions Over a Predetermined Period of Time -Scott

b) Tests that add value to Performance Testing

* How-To: How to use timers and transaction markers to evaluate performance
* How-To: How to handle hidden fields
* The Impact of Performance data source
* How-To: How to handle script and test failures
* How-To: Create Tests to Identify Points of Failure and Bottlenecks -Scott
* How-To: Create Tests to Optimize Critical User Actions -Scott
* How-To: Performance Testing Across Load Balancers (clusters).
* How-To: Execute Tests to Tune Specific Components Across Network Tiers
* How-To: Execute and Modifying Tests to Explore a Performance Bottleneck -Scott
* Presentation of VSTS solution

c) How to avoid mistakes in Performance Testing

* Biased Goals
* Unsystematic approach
* Inexperience Performance testers. Performance testing beyond test execution
* Incorrect parameters and fluctuating factors
* How-To: Translate Stakeholders Language into Real Performance Goals and Requirements -Scott
* Emulating production environments
* Not Defining concurrent users X user sessions per period of time
* How-To: Handle Outliers in Performance Test Reports -Scott
* How-To: Handle Performance Data Correctly Avoiding Over-Averaging -Scott

Microsoft Performance Analysis Methodology

a) Performance Analysis Groundwork (Goal and Benefits of performance analysis)

b) Preparing the planning performance test

c) Remove Single user performance bottleneck

* Application Performance Walkthrough (brief – Concept only)
* Application Network Analysis (brief – Concept only)

d) Stress Testing with VSTS (Basic – what to do to get the most done)

* Installing and Set up VSTS / Basics
* Core Concepts
* User Interface
* Creating Test Script
* Recording Test Script
* Manually creating Test Script
* Environment Preparation
* Hardware Setup
* Data Preparation
* Running Stress Test
* Running stress test with VSTS
* Perfmon counters
* SQL Profiler
* IIS logs
* Identify/Improve Performance Bottlenecks
* Web Bottlenecks
* SQL Bottlenecks
* 10 Common Performance Mistakes
* Capacity Planning

Data Presentation

a) How to Handle data Presentation

* Data Presentation at different levels in the organization
* How-To: How to organize efficient data graphics
* How-To: Present Averages and Percentiles of Page Response Times. -Scott
* How-To: Create Scatter and Overlaid Charts to Maximize Performance Metrics readability -Scott
* How-To: Summarize Results Across Tests Runs efficiently -Scott
* How-To: Use Degradation Curves in Reports -Scott
* How-To: Report Abandonment and Other Performance Problems -Scott
* VSTS solution aiding with reporting analysis

Performance Testing for Capacity Planning and Scalability

a) The Performance Testing strategy

* Understanding the environment
* Workload characterization planning for peaks rather than averages
* Workload forecasting
* Performance testing to aid in checking Availability and sustainability
* Performance testing to aid in analysis of Resource efficiency in Scalable systems
* The resource allocation budget in capacity planning. The testing process to aid in data collection
* Performance testing and Profiling data loads for user population for workload characterization

Topics For Later

a) Advanced Scripting for Non-Trivial Usage Scenarios

* How-To: Script Bell Curve and Sin Wave Load Patterns

Return to Performance Testing Guidance