Defrag Tools: #24 - WinDbg - Critical Sections | Defrag Tools

In this episode of Defrag Tools, Andrew Richards, Chad Beeder and Larry Larsen continue looking at the Debugging Tools for Windows (in particular WinDbg). WinDbg is a debugger that supports user mode debugging of a process, or kernel mode debugging of a computer.

This installment goes over the commands used to diagnose a Critical Section hang in a user mode application. We start with an overview of the four synchronization primitives and then delve deep in to temporary hangs, orphaned Critical Sections and deadlocks. We use these commands:

~*k
~*kv
~
~~[TID]s
!cs
!cs <pointer>
!locks

Make sure you watch Defrag Tools Episode #1 and Defrag Tools Episode #23 for instructions on how to get the Debugging Tools for Windows and how to set the required environment variables for symbols and source code resolution.

Resources:
Critical Section Objects

Timeline:
[01:00] - Hang types - CPU Looping, Temporary Hangs and Permanent Hangs
[02:00] - Synchronization Objects - Event, Semaphore, Mutex, Critical Section
[06:54] - Critical Sections
[11:45] - Debugging a Hang
[28:08] - Debugging an Orphan
[32:40] - Debugging a Deadlock

Follow the Discussion

Follow
Oops, something didn't work.

Try again?

Sign In to be begin to follow these comments

What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in. You need to be signed in to Channel 9 to use this feature.

Getting "follow" information

Stop following these comments

Start following these comments

What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in and view them all on your notifications page.

Removing your "follow"

Setting up your "follow"

Did you know you can
sign up for email notifications?
- Subscribe to this comments

edi

Jan 21, 2013 at 2:10 PM

Where is the Defrag Show - when is it coming back???

MagicAndre1981 xperf addicted

Jan 22, 2013 at 12:09 PM

Big Smile today I also debugged a Explorer crash causes by a wrong cs usage in the technet forums

0:033> !cs -s 000000001a496f50
-----------------------------------------
Critical section   = 0x000000001a496f50 (+0x1A496F50)
DebugInfo          = 0x0000000013c9bee0
LOCKED
LockCount          = 0x0
WaiterWoken        = No
OwningThread       = 0x0000000000001b04
RecursionCount     = 0x1
LockSemaphore      = 0x0
SpinCount          = 0x00000000020007d0


Stack trace for DebugInfo = 0x0000000013c9bee0:

0x000007fbd4bf0410: ntdll! ?? ::FNODOBFM::`string'+0x7E91
0x000007fbcf385be5: verifier!AVrfpInitializeCriticalSectionCommon+0x115
0x000000018012f76b: ComcastSecureBackupShareshell!DllUnregisterServer+0x12C45B
0x0000000180002a7b: ComcastSecureBackupShareshell+0x2A7B
0x000000018000302a: ComcastSecureBackupShareshell+0x302A
0x00000001801a9b4b: ComcastSecureBackupShareshell!DllUnregisterServer+0x1A683B
0x000007fbcf383b2e: verifier!AVrfpStandardDllEntryPointRoutine+0xBE
0x000007fbd4afba0e: ntdll!LdrpCallInitRoutine+0x3E
0x000007fbd4b182dc: ntdll!LdrpInitializeNode+0x192
0x000007fbd4b1787f: ntdll!LdrpInitializeGraph+0x6F
0x000007fbd4b19202: ntdll!LdrpPrepareModuleForExecution+0x1A5
0x000007fbd4b19af2: ntdll!LdrpLoadDll+0x344
0x000007fbd4b0e08b: ntdll!LdrLoadDll+0xA7
0x000007fbcf383ff6: verifier!AVrfpLdrLoadDll+0x76
0x000007fbd1aef653: KERNELBASE!LoadLibraryExW+0xBF
0x000007fbd355296c: combase!LoadLibraryWithLogging+0x2C
0x000007fbd35528c4: combase!CClassCache::CDllPathEntry::LoadDll+0x70
0x000007fbd3552693: combase!CClassCache::CDllPathEntry::Create+0x43
0x000007fbd35522a0: combase!CClassCache::CClassEntry::CreateDllClassEntry+0x196
0x000007fbd3552238: combase!CClassCache::GetClassObjectActivator+0x932

@edi

maybe Gov has no time (Windows Blue)

gt65345

Jan 23, 2013 at 7:55 AM

Would you guys consider managed code analysis in future versions of Debug Tools. I think it provides much more value for real life scenarios for people watching show since majority of applications we deal on daily basis are managed code nowdays.
windev Andrew Richards

Jan 23, 2013 at 7:26 PM

@gt65345: If you debug managed code, you'll find these fundamentals being used by the CLR to achieve the synchronization. After the next two episodes on sync objects, and handles in general, we'll tape a managed debugging episode to show the differences, and more commonly, the similarities. I actually mentioned this in next week's episode, which was taped this morning.
Thanks for the feedback.
windev Andrew Richards

Jan 23, 2013 at 7:31 PM

@MagicAndre1981: based on the limited evidence in the stack, it looks like ComcastSecureBackupShareshell is to blame. Was the CS reallocated when it was referenced?

MagicAndre1981 xperf addicted

Jan 23, 2013 at 9:34 PM

@MagicAndre1981: based on the limited evidence in the stack, it looks like ComcastSecureBackupShareshell is to blame. Was the CS reallocated when it was referenced?

I also think this was the cause. WinDbg tells me this as cause:

APPLICATION_VERIFIER_LOCKS_EXIT_THREAD_OWNS_LOCK (200)Thread cannot own a critical section.This stop is generated if a thread (thread ID is parameter1) is terminated, suspended or is in a state (worker thread finished a work item) in which it cannot hold a critical section.

The DLL is a bit older:

    Loaded symbol image file: ComcastSecureBackupShareshell.dll    Image path: C:\Program Files\SecureBackupShare\ComcastSecureBackupShareshell.dll    Image name: ComcastSecureBackupShareshell.dll    Timestamp:        Mon Nov 08 23:47:33 2010 (4CD87E05)    CheckSum:         00426F0D    ImageSize:        00450000    File version:     2.4.3.0    Product version:  2.4.0.0    File flags:       0 (Mask 3F)    File OS:          40004 NT Win32    File type:        2.0 Dll    File date:        01CCBBA5.094B4C4D    Translations:     0009.04b0    CompanyName:      Secure Backup and Share    ProductName:      Secure Backup and Share    InternalName:     x64_shell.dll    ProductVersion:   2.4.0.0    FileVersion:      2.4.3.0    FileDescription:  Secure Backup and Share Shell Extensions    LegalCopyright:   Copyright © 2010

But the user hasn't replied back if this was the fix.

LukePuplett

Jan 24, 2013 at 6:00 AM

How come, in all these decades, MSFT haven't built a debugger that's a little easier to use? Like the ANTS tools from red-gate.
JohnLudlow

Jan 24, 2013 at 6:18 AM

@gt65345: There's a pretty good show on .NET debugging with windbg http://channel9.msdn.com/Series/-NET-Debugging-Stater-Kit-for-the-Production-Environment
Won't complain if the Defrag Tools folks cover it in more detail as well though
@windev, are those crashing sample apps you have available? It might be useful to follow along and see if we get the same results (t'would confirm the debgging tools are set up properly on our boxes) and also play more with some of the scenarios you've demoed.
windev Andrew Richards

Jan 24, 2013 at 2:56 PM

@JohnLudlow: I'll ask Larry how to get them on the C9 site for download. I have no issue releasing them. Email me at defragtools@microsoft.com and I'll send you the ZIP.
windev Andrew Richards

Jan 24, 2013 at 3:00 PM

@JohnLudlow: We'll cover Managed Debugging at a high level in (near) future episode. For sure, watch the .NET Debugging series by Brad - it covers different command/techniques than what I'll go over.