Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Defrag Tools: #29 - WinDbg - ETW Logging

Download

Right click “Save as…”

In this episode of Defrag Tools, Andrew Richards, Chad Beeder and Larry Larsen continue looking at the Debugging Tools for Windows (in particular WinDbg). WinDbg is a debugger that supports user mode debugging of a process, or kernel mode debugging of a computer.

This installment goes over the Event Tracing for Windows (ETW) buffers in a kernel mode dump or live session. The ETW buffers can be extracted from the dump and viewed using the Windows Performance Toolkit (WPT). The buffers give you insight in to what has beem happening recently on the computer.

We use these commands:

  • !wmitrace.strdump
  • !wmitrace.logsave 0xNN c:\example.etl
  • !wmitrace.eventlogdump 0xNN
  • !wmitrace.help

Make sure you watch Defrag Tools Episode #1 and Defrag Tools Episode #23 for instructions on how to get the Debugging Tools for Windows and how to set the required environment variables for symbol and source code resolution. This episode shows how install the Windows Performance Toolkit.

Timeline:
[00:00] - Event Tracing for Windows (ETW)
[02:18] - Windows Performance Toolkit (WPT)
[03:48] - !wmitrace.strdump
[04:53] - !wmitrace.logsave 0xNN c:\example.etl
[05:50] - Windows Performance Analyzer (WPA) & xPerfView
[07:57] - _NT_SYMCACHE_PATH
[10:24] - !wmitrace.eventlogdump 0xNN
[12:16] - Used for logging and performance by many teams
[15:35] - Private PDBs are needed to decode some entries
[20:00] - Windows Performance Recorder (wprui.exe)
[20:35] - Disable Paging Executive
[23:40] - WPR adds the NT Kernel Logger
[24:19] - 10min run-through of the data collected with the General, CPU and Disk providers

Tags:

Follow the Discussion

  • MagicAndre1981Magic​Andre1981 xperf addicted

    Finally you touch my number 1 tool/topic ETW Big Smile

    [20:35] - Disable Paging Executive

    this is only needed for x64 Windows version to get CallStacks/Stackwalking (the data to walk the call chains is saved outside the stack)

    Adding image version data is done later when MERGING the ETL file (user mode ETW events + system data + kernel.etl into the final file).

    Without merging, you can't load the symbols on a different machine.

    [20:00] - Windows Performance Recorder (wprui.exe)

    but with the new UI you loose control on what it traced. It traces too much data and when the source system is under stress it impact the PC too much.

    I still use my scripts to run xperf with the flags I need (if I want circular log (with file size), the amount of buffers I want to use).

     

    [05:50] - Windows Performance Analyzer (WPA) & xPerfView

    that you also use xperfview shows me that all my complains during betatest where right. I said this so many times to Michael when still he was the PM of WPT/XPERF/XPERFVIEW. WPA is a terrible bad UI with blurry graphs and so many scrollbar all over the time. It sucks so much Mad

    And new users are lost in to choose which graphs they need or not.

     

    btw, UBMP = Unified Background Process Manager

  • landland

    Why I can't debug IE activex in WOW64? do you know how to debug it with windbg?

  • Andrew Richardswindev Andrew Richards

    @land: We talked about your question in #30, but here are a few tips:

    • Make sure the debugger arch matches the target (use the x86 debugger in this case)
    • Make sure you are debugging the child IE ptocess, not the parent IE process. The parent is 64bit, the children (tabs) are 32bit.
  • I may have missed something but what kind of dump does it have to be to use these commands, minidump is not supported, correct? so it has to be a full dump? I am sorry if I did not pay enough attention and missed something, the whole episode went by really fast. 

  • Andrew Richardswindev Andrew Richards

    @s3curityConsult: I don't think you missed it - pretty sure I never pointed that out. The buffers are pool memory in the kernel, so you need a kernel (2) or complete (1) dump. Kernel is the default up to win8, the win8 default is automatic, which is kernel or complete based on pagefile size. You want the c:\windows\memory.dmp file, not the c:\windows\minidump files.

  • loverboyloverboy

    Since I cannot create a .etl file with significant content, could you please post an example with lots of content inside?
    So that we can play with it using xperfview?

    Thanks in advance

  • Andrew Richardswindev Andrew Richards

    @loverboy: This script will capture a lot of cool data. You can get roughly the same data using the default options in WPRUI.exe.

    @echo off
    echo Press a key when ready to start...
    pause
    echo .
    echo ...Capturing...
    echo .
    xperf -on PROC_THREAD+LOADER+Base+Diag+Latency+FileIO+DRIVERS+DPC+DISPATCHER -stackwalk Profile+CSwitch+ReadyThread+ThreadCreate -BufferSize 1024 -MinBuffers 256 -MaxBuffers 256 -MaxFile 256 -FileMode Circular
    echo Press a key when you want to stop...
    pause
    echo .
    echo ...Stopping...
    echo .
    xperf -stop -d result.etl

  • loverboyloverboy

    Wow!
    Thanks a lot, that's what I needed ;)

    Since I have Windows 7 64bit I don't think I can use WPRUI.exe (Am I right?)
    Thanks anyway

  • loverboyloverboy

    Sorry for double posting, but what is the difference between result.etl and kernel.etl (that is bigger and automatically appears in my C:\ folder)?

  • Andrew Richardswindev Andrew Richards

    @loverboy: WPRUI works on Win7 too (not supported, but it works).
    @loverboy: kernel.etl is the kernel mode buffers, user.etl (not made here) would be the user mode buffers. The result.etl is the merge of these two, plus, it add the required information to resolve symbols. (The raw buffers just have pointers. The merge adds the module info so that offset can be mapped back to a funcion name via a symbol)

  • loverboyloverboy

    Thanks.

    Talking about WPA(and/or XPerfView) ... when you analyze on a 64bit PC a .etl taken on a 32bit machine, do you have to use the 32bit version (like windbg) or on a 64bit PC you have to use WPA or XPerfView 64bit version anyway?

  • Andrew Richardswindev Andrew Richards

    @loverboy: I always use the 64bit version for all traces - don't recall ever having an issue. If the stack includes CLR code, you won't get the function names regardless of archectural combination.

  • loverboyloverboy

    The bat file doesn't work anymore

    Now it gives an error
    C:\Program Files\Windows Performance Toolkit>Recording_Example.bat
    Press a key when ready to start...
    Premere un tasto per continuare . . .
    .
    ...Capturing...
    .
    xperf: warning: This system is not fully configured for x64 stack tracing.
    Please modify the registry under:

    HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management

    and set the value:

    DisablePagingExecutive (REG_DWORD) = 1

    Then reboot before retrying tracing.

    Note: Tracing has been enabled, this is just a warning.
    xperf: error: NT Kernel Logger: Impossibile creare un file, se il file esiste già. (0xb7).
    Press a key when you want to stop...
    Premere un tasto per continuare . . .
    .
    ...Stopping...
    .
    xperf: error: Merge ETL: Percorso specificato non valido. (0xa1).

    xperf: error: NT Kernel Logger: Impossibile creare un file, se il file esiste già. (0xb7). means Impossible creating a file if the file already exists (0xb7)

    What file is it talking about?

  • loverboyloverboy

    Sorry.
    Problem solved (I think it was just a temporary problem, since there was no result.etl file anywhere)

  • Debugging a BSOD due to a bug in Windows 8 64 bit (process MSSE a.k.a. Windows defender during quick scan, driver ndis.sys, error ATTEMPTED_EXECUTE_OF_NOEXECUTE_MEMORY, can reproduce 100% on my system).

    Have a complete memory dump created after xperf –on DiagEasy.

    !wmitrace.logsave command produces a corrupt ETL, both wpa.exe and xperfview.exe say "Trace C:\Temp\Crashes\DISK.etl could not be successfully opened [0x80070570]. Aborting operation".

    Any ideas how to fix?

  • MagicAndre1981Magic​Andre1981 xperf addicted

    @loverboy

    error 0xb7 occurs when you already run a tool which odes ETW tracing (ResMon, ProcExp):

    http://www.msfn.org/board/topic/155479-xperf-error-nt-kernel-logger-cannot-create-a-file-when-that-file-already-exists-0xb7/

  • Andrew Richardswindev Andrew Richards

    @Const: Email me (defragtools@microsoft.com) to organize a way for you to send me the dump.

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.