Defrag Tools

Defrag Tools: #14 - WinDbg - SOS

Download this episode

Download Video

Description

In this episode of Defrag Tools, Andrew Richards and Larry Larsen continue looking at the Debugging Tools for Windows (in particular WinDbg). WinDbg is a debugger that supports user mode debugging of a process, or kernel mode debugging of a computer.

This installment shows how you can view the user mode call stack and stack variables in a native, managed (.NET) or Silverlight process. We use these commands:

  • dv
  • dt
  • !sos.dumpstack
  • !sos.dumpstackobjects / !sos.dso
  • !sos.dumpobj / !sos.do
  • !sos.printexception / !sos.pe
  • .frame
  • .f+
  • .f-
  • .load
  • .unload
  • .loadby
  • .chain
  • lm / lmm / lmvm
  • .extmatch
  • .prefer_dml 1
  • .lines
  • .ecxr
  • .cls

Make sure you watch Defrag Tools Episode #1 for instructions on how to get the Debugging Tools for Windows and how to set the required environment variables for symbols and source code resolution.

Resources:
Microsoft Windows SDK for Windows 7 and .NET Framework 4
Sysinternals ProcDump
Silverlight Developer Runtime

Timeline:
[01:05] - Native vs. Managed variables
[02:35] - Display Variables (dv) and Display Type (dt)
[03:38] - Debugger Extensions (.chain, .load, .unload)
[05:43] - Extension Match (.extmatch)

[07:08] - ProcDump v5.1 captures a .NET 2 and .NET 4 exception
[08:46] - .NET engines versus .NET releases
[10:34] - Loading "Son of Strike" for .NET 2 engine applications (.loadby sos.dll mscorwks)
[13:44] - Loading "Son of Strike" for .NET 4 engine applications (.loadby sos.dll clr)
[15:24] - Dump Call Stack (!sos.dumpstack)
[16:32] - Dump Stack Objects (!sos.dumpstackobjects / !sos.dso)
[17:30] - Dump Object (!sos.dumpobject / !sos.do)
[17:51] - Enable DML (.prefer_dml 1)
[20:14] - Toggling Line display (.lines)
[20:52] - Current Frame Context (.frame, .f+, .f-); Note, registers do not change
[22:58] - ProcDump v5.1 misses Silverlight exceptions
[24:50] - Silverlight Developer Runtime (dbgshim.dll & sos.dll)
[26:10] - ProcDump v5.1 captures a Silverlight exception
[28:10] - Loading "Son of Strike" for Silverlight applications (.loadby sos.dll coreclr)
[30:47] - Missed:
Exceptions can also be displayed with !sos.printexception / !sos.pe
[31:29] - Episode review and next week... Kernel debugging

Embed

Format

Available formats for this video:

Actual format may change based on video formats available and browser capability.

    The Discussion

    • Shayan_​Anique

      Hi Very Nice Show Big Smile , But why Andrew is still using Windows 7

    • windev

      @Shayan_Anique: It was taped pre-GA (3rd Oct).  My laptop is now Windows 8.

    • Magic​Andre1981

      I'm using the PSScor DLLs instaed of SOS.

      [31:29] - Episode review and next week... Kernel debugging

      can you show ho to debug manual crash dump (http://msdn.microsoft.com/en-us/library/windows/hardware/ff545499%28v=vs.85%29.aspx) when the PC freezes?

    • Philip Saunders

      http://technet.microsoft.com/en-us/sysinternals/bb842062

      still has 17th october files ?

    • windev

      @MagicAndre1981: I didn't want to mention PSSCORx as it is a download, and not in the box.  The vast majority of debugs don't need it (over SOS). Saying that, I did use it for the last few years up until recently where some bugs in them made me go back to SOS. If you like PSSCOR, also check out SOSEx, which is a companion to SOS or PSSCORx.

      Kernel - I think Chad mentions CtrlScrollScroll and CrashOnNMI in next week's episode, and I think I might have in #1 too... If it was missed, email the show (defragtools@microsoft.com) and I'll go over them in the next kernel debugging video shoot.

      If you question is more about kernel hang debugging vs. crash debugging, we'll cover that ~6 episodes from now.

      @Philip Saunders: ProcDump v5.1 is still going through the signing process. I'll ping Mark to see if it can be expedited.

      [UPDATE: 16th Nov. 2012 -- ProcDump v5.1 is now available]

    • Magic​Andre1981

      windev wrote

      @MagicAndre1981:

      Kernel - I think Chad mentions CtrlScrollScroll and CrashOnNMI in next week's episode, and I think I might have in #1 too... If it was missed, email the show (defragtools@microsoft.com) and I'll go over them in the next kernel debugging video shoot.

      If you question is more about kernel hang debugging vs. crash debugging, we'll cover that ~6 episodes from now.

      I think you mentioned it in a show where you've explained IRQLs.

      I really think it is nice to have a full step through a hang dump of the kernel. I really want to know how to start doing this.

    • windev

      @MagicAndre1981: That is definitely a planned series of episodes. We will do that for user and kernel..

    • Magic​Andre1981

      windev wrote

      We will do that for user and kernel..

      great Smiley You should do examples of hanging (... is not responding) Explorer and Internet Explorer.

      Generic Comment Image

    • BrianHartung

      My biggest complaint about this series is you can only give it 5 stars...

    • Cyberbrad

      Hey Andrew - I am trying to dig to an application crash for a third party program. When I type lmm I see the clr module (which makes sense because Event Viewer list the crash as a .NET 4 Runtime error), so I load the .NET 4 sos.dll. But when I run the !dso command I am getting "Error requesting details - Unable to determine bounds of the gc heap" Any ideas? It is a mini dump and not a full one if that matters.

      Also, what is the difference between !clrstack and !dumpstack?

    • DakotaGeek

      For those looking for an easy approach to using "loadby" without needing to know what version of .NET you are working with, you could use a script found here:

      http://naveensrinivasan.com/2010/07/26/script-to-load-sos-within-windbg-based-on-net-framework-version/

      The script in question is:

      !for_each_module .if(($sicmp( "@#ModuleName" , "mscorwks") = 0) ) {.loadby sos mscorwks} .elsif ($sicmp( "@#ModuleName" , "clr") = 0) {.loadby sos clr}

      It probably would be best to have this saved to disk and just load it in the debugger, but I simply coppy and past from commonly used list of commands / scripts.

    • windev

      @Cyberbrad: You have to have a full dump for any managed application. Annoying - I know... especially when the application has 24Gb+ of RAM (eg. Exchange).

      !clrstack just lists the managed frames. !dumpstack lists the managed and native frames (it's the closest to 'k').

    • windev

      @DakotaGeek: Here's the (abridged) code I have in my 'preferences' extension that does the same thing as your script. I load my extension on the command line with "-a myext.dll" and auto-load SOS in DebugExtensionInitialize (or manually via !loadsos). (In my version, it also executes .prefer_dml 1, etc.)

      HRESULT Execute(IDebugClient* pDebugClient, PCSTR szCommand, BOOL bPrivate)
      {
          HRESULT hr = E_FAIL;
          // Make our own client
          IDebugClient* pDebugCreated;
          if (SUCCEEDED(hr = pDebugClient->CreateClient(&pDebugCreated)))
          {
              IDebugControl* pDebugControl;
              if (SUCCEEDED(hr = pDebugCreated->QueryInterface(__uuidof(IDebugControl), (void **)&pDebugControl)))
              {
                  if (bPrivate)
                  {
                      hr = pDebugControl->Execute(DEBUG_OUTCTL_THIS_CLIENT | DEBUG_OUTCTL_NOT_LOGGED, szCommand, DEBUG_EXECUTE_NOT_LOGGED | DEBUG_EXECUTE_NO_REPEAT);
                  }
                  else
                  {
                      hr = pDebugControl->Execute(DEBUG_OUTCTL_ALL_CLIENTS, szCommand, DEBUG_EXECUTE_NO_REPEAT);
                  }
                  pDebugControl->Release();
              }
              pDebugCreated->Release();
          }
          return hr;
      }
      
      HRESULT loadsos(IDebugClient* pDebugClient, PCSTR args)
      {
          UNREFERENCED_PARAMETER(args);
          HRESULT hr = S_OK;
          IDebugSymbols* pDebugSymbols;
          hr = pDebugClient->QueryInterface(__uuidof(IDebugSymbols), (void **)&pDebugSymbols);
          if (SUCCEEDED(hr))
          {
              ULONG nIndex = 0xFFFF;
      
              // .NET 2.0
              hr = pDebugSymbols->GetModuleByModuleName("mscorwks", 0, &nIndex, NULL);
              if ((hr == S_OK) && (nIndex !=0xFFFF))
              {
                  hr = Execute(pDebugClient, ".loadby sos mscorwks", TRUE);
                  hr = Execute(pDebugClient, ".cordll -l", TRUE);
                  goto CleanUp;
              }
      
              // .NET 4.0
              hr = pDebugSymbols->GetModuleByModuleName("clr", 0, &nIndex, NULL);
              if ((hr == S_OK) && (nIndex !=0xFFFF))
              {
                  hr = Execute(pDebugClient, ".loadby sos clr", TRUE);
                  hr = Execute(pDebugClient, ".cordll -l", TRUE);
                  goto CleanUp;
              }
      
              // Silverlight
              hr = pDebugSymbols->GetModuleByModuleName("coreclr", 0, &nIndex, NULL);
              if ((hr == S_OK) && (nIndex !=0xFFFF))
              {
                  hr = Execute(pDebugClient, ".loadby sos coreclr", TRUE);
                  hr = Execute(pDebugClient, ".cordll -l", TRUE);
                  goto CleanUp;
              }
      CleanUp:
              pDebugSymbols->Release();
          }
          return hr;
      }
      extern "C" HRESULT CALLBACK
      DebugExtensionInitialize(PULONG Version, PULONG Flags)
      {
          *Version = DEBUG_EXTENSION_VERSION(EXT_MAJOR_VER, EXT_MINOR_VER);
          *Flags = 0;  // Reserved for future use.
      
          HRESULT hr = S_OK;
          IDebugClient* pDebugClient;
          hr = DebugCreate(__uuidof(IDebugClient), (void **)&pDebugClient);
          if (hr == S_OK)
          {
              // Load SOS and do .cordll
              loadsos(pDebugClient, NULL);
              pDebugClient->Release();
          }
          return hr;
      }
      

    • Cyberbrad

      Andrew: Thanks for the info!

    • dcrearer

      Andrew... awesome demo. I have been trying to follow along, however I run into a weird problem with procdump 5.0. I have a simple divide by zero crash app (C#  .NET 4) however when it is executed procdump does not write the dump file. I have set up my environment as prescribed but no luck. Can you help a novice?

    • dcrearer

      I get the following status when using .ecxr - Unable to get exception context, HRESULT 0x8000FFFF.

      Does the .prefer_dml 1 registry command only work on dump files?

    • windev

      @dcrearer: Divide by Zero is a special case - the CPU doesn't throw them as an exception. I looked in to this briefly while making the test apps for ProcDump, and didn't come to a solution.

      @dcrearer: You can only do .ecxr if an exception context record has been added to dump by the dump creator. A hang dump (for example) won't have one.  In this case, you only have the context of when the dump was taken.

      @dcrearer: .prefer_dml 1 works in all scenarios.  It isn't dependant on live vs. dump, or user vs. kernel.  If you launch windbg via the .dmp association, then yes, the -c ".prefer_dml 1" will be executed from the command key. For manual, make a shortcut to windbg with the -c ".prefer_dml 1" added to the command line.

    • dcrearer

      Andrew... Thanks much for your feedback... Its greatly appreciated.

    • Cyberbrad

      @windev: Andrew - I just downloaded the new ProcDump 5.1 and it throws this exception when I try and run it:  "Entry Point Not Found - The procedure entry point RegGetValueW could not be located in the dynamic link library ADVAPI32.dll.".  I have tried it on 4 different Windows XP x86 PCs and each gave the same error.  They all seem to run 5.0 fine.

      I did some searching and it looks like RegGetValue() is only available on Win XP x64, Vista, and newer, so that might be the problem (MSDN link below).  It worked fine on the Win 7 x86 and x64 PCs I tried it on.  Did you update the code used to read the EulaAccepted value out of "HKEY_CURRENT_USER\Software\Sysinternals\ProcDump"?

      RegGetValue function (Windows)

      http://msdn.microsoft.com/en-us/library/windows/desktop/ms724868%28v=vs.85%29.aspx

    • windev

      @Cyberbrad: This might be by-product of the registry operations used in the new JIT support (-i <folder>). For now use v5.0 on WinXP SP3 x86 while I work on a fix. Thanks for reporting it.

    • Cyberbrad

      @windev: Will do.  Thanks

       

    • Jeff Rivett

      I just wanted to pipe up at this point and confirm that I am also seeing "Entry Point Not Found - The procedure entry point RegGetValueW could not be located in the dynamic link library ADVAPI32.dll." when I try to run procdump 5.1 on any 32 bit Windows XP machine.

      @windev: you say you're working on a fix. Any idea when that might be available? As for running procdump 5.0: I'd love to, but can't find it anywhere. Is there a sysinternals archive somewhere?

    • windev

      @Cyberbrad and @Jeff Rivett:  Mark found and fixed the bug in ProcDump v5.1 over the weekend (I used RegGetValue in the new Silverlight code). I've asked Mark to accelerate the publishing of it externally.

    • windev

      @windev:  ProcDump v5.11 got released yesterday to fix the WinXP compatibility issue.

    • windev

      @windev: ProcDump v5.13 got released today to fix another WinXP compatibility issue. ProcDump was trying to use a DbgHelp v6.1 feature when running just with DbgHelp v5.1 from WinXP's System32 folder.

    • Evginnie

      Amazing series! Thank you!

    Comments closed

    Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.