Defrag Tools #166 - Performance Analysis of UWP Apps
Play Defrag Tools #167 - Debugging User Mode Crash Dumps Redux
Description
In this episode of Defrag Tools, Andrew Richards and Chad Beeder use Debugging Tools for Windows (WinDbg) to determine the root cause of various application crashes which have occurred on Andrew's computer. We use Sysinternals ProcDump to capture the dumps.
While debugging, we take a side trip into configuring colors for Compressed and Encrypted files in Windows Explorer, and use Sysinternals Process Monitor to determine why the debugger was getting an Access Denied when loading the PDE Debugger Extension.
We did a similar investigation in these two episodes:
- Defrag Tools #135 - Debugging User Mode Crash Dumps Part 1
- Defrag Tools #136 - Debugging User Mode Crash Dumps Part 2
We cover how to install the Debugging Tools for Windows in this episode:
Get the Sysinternals tools from http://www.sysinternals.com. We use:
Get the PDE debugger extension from the Defrag Tools OneDrive
Get your Symbol Path to the Microsoft Public Symbol Server:
- Via Environment Variable
setx /m _NT_SYMBOL_PATH SRV*C:\My\Sym*https://msdl.microsoft.com/download/symbols - In the Debugger
.sympath SRV*C:\My\Sym*https://msdl.microsoft.com/download/symbols
To collect dumps of crashes on your own machine, install ProcDump as the Postmortem (AeDebugger) debugger:
md c:\dumps
procdump.exe -ma -i c:\dumps
On any dump (user or kernel), you can run automated analysis to view the issue:
!analyze -v
Debugging Cheat Sheet
- c0000005 is an Access Violation - use .ecxr & k
- c000027b is a Stowed Exception (Store Apps) - use !pde.dse
- e0434352 is a CLR Exception - use !sos.pe
- e0697282 is a C++ Exception - use .ecxr & k
- 80000003 is a Breakpoint - use !analyze -v
- When typing a decimal number, prefix it "0n"
- When typing a hexadecimal number, prefix it "0x" (the default prefix)
Common Debugger Commands
.exr -1
- View the Exception Code and the Exception Parameters
- Number looking like C0xxxxxx and 80xxxxxx are HRESULTs (Error Codes)
- Number looking like 7FFFxxxxxxxx are usually code (assembler) addresses
!address <number>
- Display the address information - Commited/Reserved/Free, Image/Mapped/Private
- Used to determine if a number is code or data.
ln <address>
- List Nearest address
- Displays the symbol at or near the address
- Used to determine if a number is code or data.
.ecxr
- Change the debugging context to the point of the exception (rather than being at the Windows Error Reporting context)
r
- View the registers at the current context. (.ecxr produces the same output)
k
- View the call stack
lmvm <module>
- View loaded module verbosely with a mask
- View a module's details, including folder, timestamp, description, copyright, product/file version
| (Vertical Bar or Pipe character)
- View the executable's path (e.g. c:\windows\notepad.exe)
!ext.error
- Get the description of an Error Code. Best at describing System Error Codes.
!pde.err
- Get the description of an Error Code. Good at describing HRESULTs (80xxxxxx and C0xxxxxx)
!pde.dpx
- Scrape the current thread for evidence (symbols, structures, strings, etc.)
.formats <number>
- Displays the number in various formats.
- Easy way of working out if a number is actually ASCII text, or a date/time
!sos.pe
- Display a CLR Exception.
- If there is an Inner Exception, click on the link to view it.
.cordll -u & .cordll -l
- If SOS isn't loaded, try to do an unload and load of the CLR support.
!peb
- View the Process Environment Block (Modules, Command Line, Environment Variables, etc.)
!teb
- View the current Thread's Environment Block (Stack Range, Last Error Code, Last Status Code, etc.)
!gle
- Get Last Error
- Display the Last Error Code and Last Status Code of the current thread
.cls
- Clear the screen.
.reload
- Force a reload (download) of symbols for the modules on the current stack.
.reload /f
- Force a full reload (download) of symbols for the modules on the current stack.
Store Applications
To view the currently installed Store Applications and their version use:
Registry Editor (regedit.exe)
- HKEY_CURRENT_USER\SOFTWARE\Classes\ActivatableClasses\Package
PowerShell
Download
More episodes in this series
Defrag Tools #168 - Powercfg Sleep Study
Related episodes
Welcome to the Inside Show
Debugging C# user code in a U-SQL query locally
Defrag Tools #172 - Application Hangs
Defrag Tools #171 - Application Insights Profiler
Defrag Tools #170 - Debugger - JavaScript Scripting
Defrag Tools #169 - Debugging Tools for Windows Team
Defrag Tools #131 - Windows 10 SDK
Defrag Tools #94 - Sysinternals Strings, FindStr, !pde.ssz
Defrag Tools: #87 - Windows 8.1 Update
Defrag Tools: #54 - IE Favorites Crash
The Discussion
-
Ytterbium What's is difference between setting these registry key vs procdump?
https://msdn.microsoft.com/en-us/library/windows/desktop/bb787181(v=vs.85).aspx
-
SOAJunK Excellent video!
But in the next one, a little bit slower will be better... ;)Thx
-
spgennard tip: you can use %* in the d.cmd script eg:
@dir %*
-
MagicAndre1981 What's is difference between setting these registry key vs procdump?
https://msdn.microsoft.com/en-us/library/windows/desktop/bb787181(v=vs.85).aspx
With WER you have more control (specify dump options, dump path per app base)
The .dumpdebug is great, but sadly not documented, I found it some months ago in this blog:
http://sww-it.ru/2016-03-13/1320#more-1320
My symbol cache is over 70GB :P (75 556 358 949 Bytes, 79271 files, 108318 folders).
-
Luke Hello,
Great show as usual!
Would you have any advise what else can be check in the debugger that would explain in more detail, why the access violation occurred in Outlook.exe(see windbg output below) on module wwlib.dll? Or would you say that updating wwlib.dll is the best approach?
0:000> |
. 0 id: 14094 examine name: C:\Program Files (x86)\Microsoft Office\Office12\OUTLOOK.EXE
0:000> .exr -1
ExceptionAddress: 5a0d7dc4 (WWLIB+0x00027dc4)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 00000000
Parameter[1]: 00000304
Attempt to read from address 00000304
0:000> .ecxr
eax=05d00f5e ebx=00000000 ecx=06acc000 edx=06acc000 esi=06acc000 edi=00000000
eip=5a0d7dc4 esp=0018ed50 ebp=0018ef60 iopl=0 nv up ei pl nz na po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00210202
WWLIB+0x27dc4:
5a0d7dc4 ffb704030000 push dword ptr [edi+304h] ds:002b:00000304=????????
0:000> kv
*** Stack trace for last set context - .thread/.cxr resets it
# ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be wrong.
00 0018ef60 5adfd382 00000000 00000001 00000000 WWLIB+0x27dc4
01 0018ef8c 30238d2e 06bb5064 06a3af64 00060c22 WWLIB+0xd4d382
02 0018efc8 3023aa63 00000000 06a598c0 00000001 OUTLOOK!FFolderSupportsUnicode+0x2487
03 0018eff0 3024bc75 00000000 06a598dc 00000000 OUTLOOK!FFolderSupportsUnicode+0x41bc
04 0018f188 3023fec4 00000000 0fffffff 06aa4800 OUTLOOK!SmoothScroll+0x1098c
05 0018f19c 3023fd55 06a598dc 0fffffff 6a6a7964 OUTLOOK!SmoothScroll+0x4bdb
06 0018f800 2f927f6c 00000000 0fffffff 00000002 OUTLOOK!SmoothScroll+0x4a6c
07 0018f884 3023aaf9 06aa4858 0fffffff 00000000 OUTLOOK!GetAllocCounters+0x2a78d
08 0018f8a4 3023cfd0 06a3abe0 06a3aba0 00000000 OUTLOOK!FFolderSupportsUnicode+0x4252
09 0018f910 3023c556 06aa3c30 0018f940 02702970 OUTLOOK!SmoothScroll+0x1ce7
0a 0018f964 2f976570 06aa3c30 00000000 00000000 OUTLOOK!SmoothScroll+0x126d
0b 0018f988 30267a8e 06aa3c30 00000000 07150e60 OUTLOOK!GetCentralObject+0x2384
0c 0018fa40 301a2cfd 00000000 0000001b 0018fa98 OUTLOOK!SmoothScroll+0x2c7a5
0d 0018fa50 2f987ab1 02702400 3028e990 00000001 OUTLOOK!HrMsgDownloadedNotification+0x2dbc6
0e 0018fa98 2f9a6ffc 0661bc88 00000000 00000001 OUTLOOK!XGetExplorerStoragePath+0x4adb
0f 0018fad4 3026e830 00000001 00000000 00000001 OUTLOOK!XGetExplorerStoragePath+0x24026
10 0018faf8 3026ea50 00000000 00000000 30290b38 OUTLOOK!SmoothScroll+0x33547
11 0018fb34 2f973397 00000000 00000000 00000000 OUTLOOK!SmoothScroll+0x33767
12 0018fb5c 2f80ec5f 2f7f0000 00000000 00384c6e OUTLOOK!GetAllocCounters+0x75bb8
13 0018fbec 757c338a 7efde000 0018fc38 77869902 OUTLOOK+0x1ec5f
14 0018fbf8 77869902 7efde000 7a58ce6c 00000000 kernel32!BaseThreadInitThunk+0xe (FPO: [Non-Fpo])
15 0018fc38 778698d5 2f7f3910 7efde000 ffffffff ntdll!__RtlUserThreadStart+0x70 (FPO: [Non-Fpo])
16 0018fc50 00000000 2f7f3910 7efde000 00000000 ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])
0:000> lmvm WWLIB
Browse full module list
start end module name
5a0b0000 5b1c4000 WWLIB T (no symbols)
Loaded symbol image file: WWLIB.DLL
Image path: C:\Program Files (x86)\Microsoft Office\Office12\WWLIB.DLL
Image name: WWLIB.DLL
Browse all global symbols functions data
Timestamp: Thu May 19 04:51:24 2016 (573D383C)
CheckSum: 01102DB5
ImageSize: 01114000
File version: 12.0.6749.5000
Product version: 12.0.6749.0
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4 -
ChadBeeder @Luke:Not enough information there to determine anything. If you share the .dmp on a OneDrive we could take a look. If I'm not mistaken, though, that looks like Office 2007, which hasn't gotten any non-security fixes in several years now, so the real answer might be to update to a newer version of Office.
-
Joyce Gammill Totally lost; not a clue what y'all are talking about (senior citizen, duh - frustrating); want the update (do I?), but not sure I'll ever get it, i.e., understand what I should be doing. Got anything plain and simple for old gray mares, maybe? FYI: Dell Insipron 1545 laptop, but never used as such - plugged in, no battery, never leaves desk.
-
Timothy Suhr I was able to follow the direction the video made. It would be helpful to have more information about this hole topic. Please do not stop here! As a tester I think this may help. More about what would best go into bug reports. Also the use of these tools from a testing perspective would be helpful.
Thanks again and keep up the great work... -
pendingio Thanks for the Video guys.
Any Details when this book is coming out, they keep pushing it back a month :)
https://www.microsoftpressstore.com/store/troubleshooting-with-the-windows-sysinternals-tools-9780735684447 -
matt if i want to get the most inclusive dump, should i use ProcDump -ma or -mp?
-
matthew grossman NM -ma it is
-
siodmy how do you get !pe right away? I have to run .cordll -l to make windbg load sos.
-
adolfox Just discovered your show and am super excited about it. Thank you for making it!