Sign In
Home
Recent
Media
Videos
Podcasts
Screencasts
Shows
ARCast with Ron Jacobs
ARCast.TV
Behind The Code
See All Shows...
Forums
Coffeehouse
Tech Off
Feedback
Sandbox
Key Topics
Visual Studio
Continuum
Wiki
Search
About
Subscribe
performancetestingtableofcontents
Cancel
Save
Edit
Return to
Performance Testing Guidance
Table of Contents - Performance Testing Guidance
This page shows the current working version of the Table of Contents.
Introduction to Performance Testing
a) Introduce Key Terms and Concepts
* Explain or Principle:
Types of Performance Testing
-Scott
* Explain or Principle:
Risks Addressed via Performance Testing
-Scott
* Explain or Principle:
Understanding Volume
-Scott
* Explain or Principle:
Test tools, test artifacts
b) Performance Test Strategy
* How-To:
Evaluate Systems to Improve Performance Testing Effectiveness
-Scott
* How-To:
Determine Performance Testing Objectives, Building the Targets
-Scott
* How-To:
Quantify End Users Response Time Goals
-Scott
*
Test assets
* How-To:
Baseline Application Performance
-Scott
*
Test Resources tools and personas. Who is the Performance tester?
* How-To:
Manage the Performance Test cycle. (Agile)
-Scott
* How-To:
Manage the Performance Test cycle. (CMMI)
-Scott
* How-To:
Conduct a Performance Testing Risk Assessment
-Scott
c) Environment - Performance Test Beds
*
Creating
*
Troubleshooting
*
Real world simulation
*
Server configurations
*
Client Configurations
*
Scalable test environments
d) Key Variables and Metrics
* How-To:
Workload Characterization
* How-To:
Client variables and metrics (Location, network connection, payloads)
* How-To:
Server variables and metrics (Interaction speed, latency, software hardware configurations)
* How-To:
Load Patterns in Web Applications
* How-To:
Server Performance Metrics: data collection and monitoring
e) Test Execution
* How-To:
VSTS solution
* How-To:
Test Scripts (creating troubleshooting)
* How-To:
Reporting and Analysis
* How-To:
Data Presentations
Defining the Performance and Stress Testing
a) Modeling the User experience
* How-To:
Simulate Realistic User Delays
-Scott
* How-To:
Model Representative User Groups
-Scott
* How-To:
Simulate Realistic User Patterns
-Scott
* How-To:
Account for User Abandonment
-Scott
* How-To:
Define User Sessions Over a Predetermined Period of Time
-Scott
b) Tests that add value to Performance Testing
* How-To:
How to use timers and transaction markers to evaluate performance
* How-To:
How to handle hidden fields
*
The Impact of Performance data source
* How-To:
How to handle script and test failures
* How-To:
Create Tests to Identify Points of Failure and Bottlenecks
-Scott
* How-To:
Create Tests to Optimize Critical User Actions
-Scott
* How-To:
Performance Testing Across Load Balancers (clusters).
* How-To:
Execute Tests to Tune Specific Components Across Network Tiers
* How-To:
Execute and Modifying Tests to Explore a Performance Bottleneck
-Scott
*
Presentation of VSTS solution
c) How to avoid mistakes in Performance Testing
*
Biased Goals
*
Unsystematic approach
*
Inexperience Performance testers. Performance testing beyond test execution
*
Incorrect parameters and fluctuating factors
* How-To:
Translate Stakeholders Language into Real Performance Goals and Requirements
-Scott
*
Emulating production environments
*
Not Defining concurrent users X user sessions per period of time
* How-To:
Handle Outliers in Performance Test Reports
-Scott
* How-To:
Handle Performance Data Correctly Avoiding Over-Averaging
-Scott
Microsoft Performance Analysis Methodology
a) Performance Analysis Groundwork (Goal and Benefits of performance analysis)
b) Preparing the planning performance test
c) Remove Single user performance bottleneck
*
Application Performance Walkthrough (brief – Concept only)
*
Application Network Analysis (brief – Concept only)
d) Stress Testing with VSTS (Basic – what to do to get the most done)
* Installing and Set up VSTS / Basics
*
Core Concepts
*
User Interface
*
Creating Test Script
*
Recording Test Script
*
Manually creating Test Script
* Environment Preparation
*
Hardware Setup
*
Data Preparation
* Running Stress Test
*
Running stress test with VSTS
*
Perfmon counters
*
SQL Profiler
*
IIS logs
* Identify/Improve Performance Bottlenecks
*
Web Bottlenecks
*
SQL Bottlenecks
*
10 Common Performance Mistakes
* Capacity Planning
Data Presentation
a) How to Handle data Presentation
*
Data Presentation at different levels in the organization
* How-To:
How to organize efficient data graphics
* How-To:
Present Averages and Percentiles of Page Response Times.
-Scott
* How-To:
Create Scatter and Overlaid Charts to Maximize Performance Metrics readability
-Scott
* How-To:
Summarize Results Across Tests Runs efficiently
-Scott
* How-To:
Use Degradation Curves in Reports
-Scott
* How-To:
Report Abandonment and Other Performance Problems
-Scott
*
VSTS solution aiding with reporting analysis
Performance Testing for Capacity Planning and Scalability
a) The Performance Testing strategy
*
Understanding the environment
*
Workload characterization planning for peaks rather than averages
*
Workload forecasting
*
Performance testing to aid in checking Availability and sustainability
*
Performance testing to aid in analysis of Resource efficiency in Scalable systems
*
The resource allocation budget in capacity planning. The testing process to aid in data collection
*
Performance testing and Profiling data loads for user population for workload characterization
Topics For Later
a) Advanced Scripting for Non-Trivial Usage Scenarios
* How-To:
Script Bell Curve and Sin Wave Load Patterns
Return to
Performance Testing Guidance
Return to %5bPerformance Testing Guidance%7cPerformanceTestingGuidance%5d ---- %21%21 Table of Contents - Performance Testing Guidance This page shows the current working version of the Table of Contents. ---- %21%21%21 Introduction to Performance Testing a%29 Introduce Key Terms and Concepts * Explain or Principle%3a %5bTypes of Performance Testing%7cPerformanceTestingTestTypeTerms%5d _-Scott_ * Explain or Principle%3a %5bRisks Addressed via Performance Testing%7cPerformanceTestingScaleTerms%5d _-Scott_ * Explain or Principle%3a %5bUnderstanding Volume%7cPerformanceTestingSessionsTerms%5d _-Scott_ * Explain or Principle%3a %5bTest tools%2c test artifacts%7cPerformanceTestingToolTerms%5d b%29 Performance Test Strategy * How-To%3a %5bEvaluate Systems to Improve Performance Testing Effectiveness%7cPerformanceTestingEvaluateStrategy%5d _-Scott_ * How-To%3a %5bDetermine Performance Testing Objectives%2c Building the Targets%7cPerformanceTestingObjectivesStrategy%5d _-Scott_ * How-To%3a %5bQuantify End Users Response Time Goals%7cPerformanceTestingPsychologyStrategy%5d _-Scott_ * %5bTest assets%7cPerformanceTestingAssetsStrategy%5d * How-To%3a %5bBaseline Application Performance%7cPerformanceTestingBaselineStrategy%5d _-Scott_ * %5bTest Resources tools and personas. Who is the Performance tester%3f%7cPerformanceTestingResourceStrategy%5d * How-To%3a %5bManage the Performance Test cycle. %28Agile%29%7cPerformanceTestingScheduleAgileStrategy%5d _-Scott_ * How-To%3a %5bManage the Performance Test cycle. %28CMMI%29%7cPerformanceTestingScheduleCMMIStrategy%5d _-Scott_ * How-To%3a %5bConduct a Performance Testing Risk Assessment%7cPerformanceTestingRiskStrategy%5d _-Scott_ c%29 Environment - Performance Test Beds * %5bCreating%7cPerformanceTestingCreatingBeds%5d * %5bTroubleshooting%7cPerformanceTestingTroubleshootingBeds%5d * %5bReal world simulation%7cPerformanceTestingRealWorldBeds%5d * %5bServer configurations%7cPerformanceTestingServerBeds%5d * %5bClient Configurations%7cPerformanceTestingClientBeds%5d * %5bScalable test environments%7cPerformanceTestingScalableBeds%5d d%29 Key Variables and Metrics * How-To%3a %5bWorkload Characterization%7cPerformanceTestingWorkloadVariables%5d * How-To%3a %5bClient variables and metrics %28Location%2c network connection%2c payloads%29%7cPerformanceTestingClientVariables%5d * How-To%3a %5bServer variables and metrics %28Interaction speed%2c latency%2c software hardware configurations%29%7cPerformanceTestingServerVariables%5d * How-To%3a %5bLoad Patterns in Web Applications%7cPerformanceTestingLoadMetrics%5d * How-To%3a %5bServer Performance Metrics%3a data collection and monitoring%7cPerformanceTestingServerMetrics%5d e%29 Test Execution * How-To%3a %5bVSTS solution%7cPerformanceTestingVSTSExecution%5d * How-To%3a %5bTest Scripts %28creating troubleshooting%29%7cPerformanceTestingScriptsExecution%5d * How-To%3a %5bReporting and Analysis%7cPerformanceTestingReportingExecution%5d * How-To%3a %5bData Presentations%7cPerformanceTestingDataExecution%5d %21%21%21 Defining the Performance and Stress Testing a%29 Modeling the User experience * How-To%3a %5bSimulate Realistic User Delays%7cPerformanceTestingUserDelays%5d _-Scott_ * How-To%3a %5bModel Representative User Groups%7cPerformanceTestingUserGroups%5d _-Scott_ * How-To%3a %5bSimulate Realistic User Patterns%7cPerformanceTestingUserPatterns%5d _-Scott_ * How-To%3a %5bAccount for User Abandonment%7cPerformanceTestingUserAbandonment%5d _-Scott_ * How-To%3a %5bDefine User Sessions Over a Predetermined Period of Time%7cPerformanceTestingUserSessions%5d _-Scott_ b%29 Tests that add value to Performance Testing * How-To%3a %5bHow to use timers and transaction markers to evaluate performance%7cPerformanceTestingTimerValue%5d * How-To%3a %5bHow to handle hidden fields%7cPerformanceTestingHiddenValue%5d * %5bThe Impact of Performance data source%7cPerformanceTestingDataValue%5d * How-To%3a %5bHow to handle script and test failures%7cPerformanceTestingFailureValue%5d * How-To%3a %5bCreate Tests to Identify Points of Failure and Bottlenecks%7cPerformanceTestingBottleneckValue%5d _-Scott_ * How-To%3a %5bCreate Tests to Optimize Critical User Actions%7cPerformanceTestingOptimizeValue%5d _-Scott_ * How-To%3a %5bPerformance Testing Across Load Balancers %28clusters%29.%7cPerformanceTestingLoadBalancerValue%5d * How-To%3a %5bExecute Tests to Tune Specific Components Across Network Tiers%7cPerformanceTestingNetworkValue%5d * How-To%3a %5bExecute and Modifying Tests to Explore a Performance Bottleneck%7cPerformanceTestingExploitValue%5d _-Scott_ * %5bPresentation of VSTS solution%7cPerformanceTestingVSTSValue%5d c%29 How to avoid mistakes in Performance Testing * %5bBiased Goals%7cPerformanceTestingBiasMistake%5d * %5bUnsystematic approach%7cPerformanceTestingUnsystematicMistake%5d * %5bInexperience Performance testers. Performance testing beyond test execution%7cPerformanceTestingInexperienceMistake%5d * %5bIncorrect parameters and fluctuating factors%7cPerformanceTestingIncorrectParametersMistake%5d * How-To%3a %5bTranslate Stakeholders Language into Real Performance Goals and Requirements%7cPerformanceTestingTranslationMistake%5d _-Scott_ * %5bEmulating production environments%7cPerformanceTestingEmulationMistake%5d * %5bNot Defining concurrent users X user sessions per period of time%7cPerformanceTestingConcurrentUsersMistake%5d * How-To%3a %5bHandle Outliers in Performance Test Reports%7cPerformanceTestingOutliersMistake%5d _-Scott_ * How-To%3a %5bHandle Performance Data Correctly Avoiding Over-Averaging%7cPerformanceTestingAveragingMistake%5d _-Scott_ %21%21%21 Microsoft Performance Analysis Methodology a%29 Performance Analysis Groundwork %28Goal and Benefits of performance analysis%29 b%29 Preparing the planning performance test c%29 Remove Single user performance bottleneck * %5bApplication Performance Walkthrough %28brief – Concept only%29%7cPerformanceTestingSingleBottleneckConcept%5d * %5bApplication Network Analysis %28brief – Concept only%29%7cPerformanceTestingNetworkBottleneckConcept%5d d%29 Stress Testing with VSTS %28Basic – what to do to get the most done%29 * Installing and Set up VSTS / Basics * %5bCore Concepts%7cPerformanceTestingConceptsVSTS%5d * %5bUser Interface%7cPerformanceTestingInterfaceVSTS%5d * %5bCreating Test Script%7cPerformanceTestingCreateScriptVSTS%5d * %5bRecording Test Script%7cPerformanceTestingRecordScriptVSTS%5d * %5bManually creating Test Script%7cPerformanceTestingManualVSTS%5d * Environment Preparation * %5bHardware Setup%7cPerformanceTestingHardwarePrep%5d * %5bData Preparation%7cPerformanceTestingDataPrep%5d * Running Stress Test * %5bRunning stress test with VSTS%7cPerformanceTestingVSTSStress%5d * %5bPerfmon counters%7cPerformanceTestingPerfmonStress%5d * %5bSQL Profiler%7cPerformanceTestingSQLStress%5d * %5bIIS logs%7cPerformanceTestingIISStress%5d * Identify/Improve Performance Bottlenecks * %5bWeb Bottlenecks%7cPerformanceTestingWebBottleneck%5d * %5bSQL Bottlenecks%7cPerformanceTestingSQLBottleneck%5d * %5b10 Common Performance Mistakes%7cPerformanceTesting10Mistakes%5d * Capacity Planning %21%21%21 Data Presentation a%29 How to Handle data Presentation * %5bData Presentation at different levels in the organization%7cPerformanceTestingLevelsPresentation%5d * How-To%3a %5bHow to organize efficient data graphics%7cPerformanceTestingGraphicPresentation%5d * How-To%3a %5bPresent Averages and Percentiles of Page Response Times. %7cPerformanceTestingAveragingPresentation%5d _-Scott_ * How-To%3a %5bCreate Scatter and Overlaid Charts to Maximize Performance Metrics readability%7cPerformanceTestingScatterPresentation%5d _-Scott_ * How-To%3a %5bSummarize Results Across Tests Runs efficiently%7cPerformanceTestingSummaryPresentation%5d _-Scott_ * How-To%3a %5bUse Degradation Curves in Reports%7cPerformanceTestingDegredationPresentation%5d _-Scott_ * How-To%3a %5bReport Abandonment and Other Performance Problems%7cPerformanceTestingAbandonmentPresentation%5d _-Scott_ * %5bVSTS solution aiding with reporting analysis%7cPerformanceTestingVSTSPresentation%5d %21%21%21 Performance Testing for Capacity Planning and Scalability a%29 The Performance Testing strategy * %5bUnderstanding the environment%7cPerformanceTestingEnvironmentScale%5d * %5bWorkload characterization planning for peaks rather than averages%7cPerformanceTestingPlanningScale%5d * %5bWorkload forecasting%7cPerformanceTestingForcastScale%5d * %5bPerformance testing to aid in checking Availability and sustainability%7cPerformanceTestingAvailabilityScale%5d * %5bPerformance testing to aid in analysis of Resource efficiency in Scalable systems%7cPerformanceTestingResourceScale%5d * %5bThe resource allocation budget in capacity planning. The testing process to aid in data collection%7cPerformanceTestingBudgentScale%5d * %5bPerformance testing and Profiling data loads for user population for workload characterization%7cPerformanceTestingDataScale%5d ---- %21%21%21 Topics For Later a%29 Advanced Scripting for Non-Trivial Usage Scenarios * How-To%3a %5bScript Bell Curve and Sin Wave Load Patterns%7cPerformanceTestingAdvScriptBell%5d ---- Return to %5bPerformance Testing Guidance%7cPerformanceTestingGuidance%5d
HTML
Preview
Cancel
Save
Edit
Delete
Edit
Comment on Post
Delete
Edit
Reply
In reply to {0}
Download:
[Pending]
Delete
Edit
Reply
In reply to {0}
Download:
[Pending]
Delete
Edit
Reply
In reply to {0}
Download:
[Pending]
Delete
Edit
Reply
In reply to {0}
Download:
[Pending]
Delete
Edit
Reply
In reply to {0}
Download:
[Pending]
Delete
Edit
Reply
In reply to {0}
Download:
[Pending]
Delete
Edit
Reply
In reply to {0}
Download:
[Pending]
Delete
Edit
Reply
In reply to {0}
Download:
[Pending]
Delete
Edit
Reply
In reply to {0}
Download:
[Pending]
Delete
Edit
Comment on Post
Be the first to comment!
Reply
Reply to root
In reply to {0}
Start related discussion
Forum:
PDC Talk
Site Feedback
Tech Off
The 9 Guy Around The World
The Coffeehouse
Subject:
Design
HTML
Preview
Tags
*Loading available tags