Defrag Tools: #44 - WPT - DiskIO Analysis

In this episode of Defrag Tools, Andrew Richards, Chad Beeder and Larry Larsen continue walking you through the Windows Performance Toolkit (WPT). Example xPerf scripts.
Resources:
Defrag Tools: #23 - Windows 8 SDK
Defrag Tools: #29 - WinDbg - ETW Logging
Windows Performance Analysis Developer Center
Windows Performance Toolkit
Channel 9 Videos
NTDebugging Blog Article
PFE Blog Series
Timeline:
File
[00:00] - Process Monitor vs. WPT
[01:48] - xperf -on PROC_THREAD+LOADER+FILENAME+FILE_IO+FILE_IO_INIT -stackwalk ...
[03:43] - Process Monitor design (I asked Mark; filtering is done in User Mode)
[05:25] - WPA - File Analysis
[09:42] - Comparison to Process Monitor "Enable Advanced Output"
Registry
[16:47] - xperf -on PROC_THREAD+LOADER+REGISTRY -stackwalk ...
[18:25] - WPR Profiles (FileIO & Registry)
[20:50] - WPA - Registry Analysis
Registry Hive
[25:55] - xperf -on PROC_THREAD+LOADER+REG_HIVE -stackwalk ...
[28:22] - Logoff/Logon to show Registry Hive unload/load
[29:10] - WPA - Registry Hive Analysis
Summary
[33:16] - Summary
Example: "xperf - Collect FileIO.cmd"
@echo off
echo Press a key when ready to start...
pause
echo .
echo ...Capturing...
echo .
xperf -on PROC_THREAD+LOADER+FILENAME+FILE_IO+FILE_IO_INIT -stackwalk FileCreate+FileCleanup+FileClose+FileRead+FileWrite+FileSetInformation+FileDelete+FileRename+FileDirEnum+FileFlush+FileQueryInformation -BufferSize 1024 -MinBuffers 256 -MaxBuffers 256 -MaxFile 256 -FileMode Circular
echo Press a key when you want to stop...
pause
echo .
echo ...Stopping...
echo .
xperf -stop -d fileio.etl
Example: "xperf - Collect Registry.cmd"
@echo off
echo Press a key when ready to start...
pause
echo .
echo ...Capturing...
echo .
xperf -on PROC_THREAD+LOADER+REGISTRY -stackwalk RegQueryKey+RegEnumerateKey+RegEnumerateValueKey+RegDeleteKey+RegCreateKey+RegOpenKey+RegSetValue+RegDeleteValue+RegQueryValue+RegQueryMultipleValue+RegSetInformation+RegFlush+RegKcbCreate+RegKcbDelete+RegVirtualize+RegCloseKey -BufferSize 1024 -MinBuffers 256 -MaxBuffers 256 -MaxFile 256 -FileMode Circular
echo Press a key when you want to stop...
pause
echo .
echo ...Stopping...
echo .
xperf -stop -d registry.etl
Example: "xperf - Collect RegHive.cmd"
@echo off
echo Press a key when ready to start...
pause
echo .
echo ...Capturing...
echo .
xperf -on PROC_THREAD+LOADER+REG_HIVE -stackwalk RegHiveInit+RegHiveDestroy+RegHiveLink+RegHiveDirty -BufferSize 1024 -MinBuffers 256 -MaxBuffers 256 -MaxFile 256 -FileMode Circular
echo Press a key when you want to stop...
pause
echo .
echo ...Stopping...
echo .
xperf -stop -d reghive.etl
nice show.
With registry, there is an important change in Windows 8:
Saving application registry changes on Windows 8
http://support.microsoft.com/kb/2784761/en-us
To maximize performance, updates to the registry in Windows 8 and Windows Server 2012 are not immediately flushed to disk. Instead, the registry flushes modified registry data to the disk at regular intervals of time. In addition, modified registry data is saved to disk when the system shuts down. In most cases, these mechanisms are sufficient to ensure that registry modifications safely reach the disk.
Because registry changes are not immediately flushed to disk, if a machine loses power immediately after an application modifies the registry, the application's registry changes may not be saved. If this occurs, the application may observe the following effects when the system restarts:
- Registry changes made by the application may not be visible
- A newly installed driver may no longer appear to be installed, and will need to be reinstalled
- A newly uninstalled driver will still be installed, and need to be uninstalled again
I think this is important to mention here.
Not directly related to this video, but in general.
Whenever I launch those cmd, xperf correctly warns me that "This system is not fully configured for x64 stack tracing" so that Disable Paging Executive must be set at 1, to have valid results.
My question is: "Why isn't Disable Paging Executive set to 1 as default in Windows 7?"
I have Windows 7 Home Premium 64bit with 16 GB RAM
What do I risk if I leave it set at 1 as default?
is there any (brief) technical reason why Microsoft didn't leave at 1 in W7, while I understand it is set at 1 in W8?
I've tried using xperf for various things before, but the problem I run into is that the amount of data generated in the output files is huge--so using xperf seems to be limited to very small time frames.
My use case of interest is tracing/monitoring what happens over the course of a build (entire process tree, files read, ... etc), which could last over an hour. The kind of data I'd look for, is what can be had with strace on *nix, but there appears to be no user-configurable way to filter at that level of granularity. Can you offer any tips?
@garenp
use ProcMon for this tracing instead.
Very nice!!!
@MagicAndre1981: ProcMon has the same problem--it starts filling up a log file that gets really huge.
you can filter ProcMon easier. You can also drop filtered events to make the file smaller. That's why ProcMon is better for you.
@MagicAndre1981: ProcMon has the same problem: the filtering can't be done until *after* you create a huge dump of data. I need to apply filtering *before* the dump ever gets stored, because the data is just too vast. 1-2 hour builds generate *way* too much data to filter them after-the-fact.
no, you can apply the filter before satrting to log and set "dropped filtered events". This makes the trace smaller.
Does anybody know why or can confirm the limited capability of File I/O stacks captured on Windows 7x64, or Windows Server 2008R2 ?
It seems to work fine on Windows 8 and Server 2012, which can obviously also be seen in the above Video. So it's become quite frustrating.
To be clear, I'm talking specifically about when I add the Stack column to "File I/O Activity by Process, Thread, Type", switch to table view, then add\adjust the columns [*A].
Symbols resolved, x64 registry setting, etc. See below.
But I still only get "?!?" frames.
[*A] Columns I typically use are in this order:
Line#, Process, Event Type, Event Sub Type, FileName, Thread, Stack | Gold bar| etc ...
Curiously, I do seem some stacks, but only under "System Activity", which is not even close to what I want.
DETAILS
Scenario 1:
VMWare Player 5.0.2 build-1031769
Line # | Configuration | Value |
3 | Product Name | Windows Server 2008 R2 Enterprise |
4 | Build Lab | 7600.16385.amd64fre.win7_rtm.090713-1255 |
5 | OS Version | 6.1 |
6 | Build | 7600 |
7 | Number of Processors | 2 |
8 | Processor Speed | 3392 MHz |
9 | Hyper-Threading Enabled Processors | 0x0000000000000000 |
10 | Memory Size | 2048 MB |
11 | Page Size | 4096 Bytes |
12 | Allocation Granularity | 65536 Bytes |
13 | Supported Power States | S1 S4 S5 |
14 | Boot Drive | Disk 0 - Drive C - NTFS |
15 | ETW Internal Version | 25 |
Scenario 2:Real PC - HP8200Elite (Hosting Scenario 1)
Line # | Configuration | Value |
3 | Product Name | Windows 7 Enterprise |
4 | Build Lab | 7601.18113.amd64fre.win7sp1_gdr.130318-1533 |
5 | OS Version | 6.1 |
6 | Build | 7601 |
7 | Number of Processors | 8 |
8 | Processor Speed | 3392 MHz |
9 | Hyper-Threading Enabled Processors | 0x00000000000000FF |
10 | Memory Size | 16342 MB |
11 | Page Size | 4096 Bytes |
12 | Allocation Granularity | 65536 Bytes |
13 | Supported Power States | S3 S4 S5 |
14 | Boot Drive | Disk 0 - Drive C - NTFS |
15 | ETW Internal Version | 25 |
Scenario 3 (Working fine, so this is FYI):
Line # | Configuration | Value |
3 | Product Name | Windows Server 2012 Standard |
4 | Build Lab | 9200.16581.amd64fre.win8_gdr.130410-1505 |
5 | OS Version | 6.2 |
6 | Build | 9200 |
7 | Number of Processors | 2 |
8 | Processor Speed | 2667 MHz |
9 | Hyper-Threading Enabled Processors | 0x0000000000000000 |
10 | Memory Size | 2048 MB |
11 | Page Size | 4096 Bytes |
12 | Allocation Granularity | 65536 Bytes |
13 | Supported Power States | S1 S4 S5 |
14 | Boot Drive | Disk 0 - Drive C - NTFS |
15 | ETW Internal Version | 42 |
Things tried so far:
1. Set and verified registry setting. Restarted many time since:
reg add "HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management" /v DisablePagingExecutive /t REG_DWORD /d 1
2. Using Andrew's Fileio.cmd. As in:
... xperf -on PROC_THREAD+LOADER+FILENAME+FILE_IO+FILE_IO_INIT -stackwalk FileCreate+FileCleanup+FileClose+FileRead+FileWrite -BufferSize 1024 -MinBuffers 256 -MaxBuffers 256 -MaxFile 256 -FileMode Circular ...
3. Using WPR with CPU and File IO enabled.
4. Switched to Windows 8 x64 VM and saw it works fine - as per my original expectation on Windows 7x