Defrag Tools: #88 - Symbol Folder Hierarchy - index2.txt

Download this episode

Download Video

Download captions

Download Captions

Description

In this episode of Defrag Tools, Andrew Richards and Chad Beeder talk about how you can change the Symbol folder's hierarchy to be 2-Tier instead of 1-Tier - by adding an index2.txt file to the root.

Symbol Folder Hierarchies:
1-Tier : c:\My\Sym\<Filename>\<Index>\<Filename>
e.g.
c:\My\Sym\ntdll.pdb\B17873D46FFF421587F6E743D1A4B1851\ntdll.pdb
c:\My\Sym\mshtml.pdb\40B6834E5E2B4354A1872B1236917D6A2\mshtml.pdb


2-Tier : c:\My\Sym\<First 2 letters of Filename>\<Filename>\<Index>\<Filename>
e.g.
c:\My\Sym\nt\ntdll.pdb\B17873D46FFF421587F6E743D1A4B1851\ntdll.pdb
c:\My\Sym\ms\mshtml.pdb\40B6834E5E2B4354A1872B1236917D6A2\mshtml.pdb

c:\My\Sym\index2.txt

Timeline:
[00:00] - Soda fridge
[01:16] - NTFS Compression (compact.exe)
[02:12] - 1-Tier Symbol Folder Hierarchy
[03:08] - 2-Tier Symbol Folder Hierarchy
[03:16] - Make index2.txt in root
[03:58] - [Why did I say 27 not 26? -- I included the space/omission scenario]
[04:53] - Echo command to make the index2.txt file
[05:27] - Use PowerShell/VBScript to convert a 1-Tier to 2-Tier structure
[06:45] - Recommended for new or large installations
[08:15] - Email us your issues at defragtools@microsoft.com

Environment Variables - Symbols.cmd

md c:\My
md c:\My\Sym
md c:\My\SymCache
compact /c /s /i /q c:\My\Sym\
compact /c /s /i /q c:\My\SymCache\
setx /m DBGHELP_HOMEDIR C:\My
setx /m _NT_SYMBOL_PATH SRV*C:\My\Sym*http://msdl.microsoft.com/download/symbols
setx /m _NT_SYMCACHE_PATH C:\My\SymCache
echo Hello > c:\My\Sym\index2.txt

Embed

Format

Available formats for this video:

Actual format may change based on video formats available and browser capability.

    The Discussion

    • User profile image
      Magic​Andre1981

      I read about it a few years ago:

      http://voneinem-windbg.blogspot.de/2008/12/symbol-server-performance-improvement.html

      It also points to a tool which does the conversation.

    • User profile image
      Dono

      Thanks for another useful post. Rather than creating a new file with the word "Hello" in it, I changed my script to create an empty file (0 bytes): type nul > c:\My\Sym\index2.txt (Note that echo. > c:\My\Sym\index2.txt will create a file that is not truly empty.)

      While it appears to be working, I think that there is a small issue with the compact command. Specifically, the /s switch. This will compact the current folder, not necessarily the specified folder. I think that this should be changed to the following:

      compact /c /i /q /s:c:\My\Sym\
      compact /c /i /q /s:c:\My\SymCache\

      Until making this change, on my machine the original commands also enabled compression on several other unrelated folders that I also had. See here for details: http://technet.microsoft.com/en-us/library/cc976800.aspx

       

    • User profile image
      aallidina

      2-tier symbols is a cool idea but it doesn't seem to work with Visual Studio, in particular VS 2013. I took a look with Procmon:

      When VS goes to look for a symbol, it does not look for index2.txt. It *does* look for a file called flat.txt.

      If flat.txt is not present, VS does a regular 1-tier lookup for the symbol
      ex: c:\Symbols\advapi32.pdb\699F5B1DC86D40EA9280C58CB9AA46172

      If flat.txt is present, VS does a flat lookup for the symbol
      ex: c:\symbols\advapi32.pdb

      In neither case do we see 2-tier support. I would love it if someone could prove me wrong!

    • User profile image
      windev

      @aallidina: Well that's annoying. What location is symsrv.dll being loaded from (looking in the Process Monitor log).

      The versions in system32 isn't the full version. You need to use the version that comes with the debugger (same story goes for dbghelp.dll).

    • User profile image
      aallidina

      @windev:It uses:

      C:\Program Files (x86)\Common7\IDE\symsrv.dll. 
      Version is 6.3.9600.17054

      C:\Program Files (x86)\Common7\IDE\Remote Debugger\x64\dbghelp.dll
      Version is 6.3.9600.16384

      Would you try replacing them with the latest versions?

    • User profile image
      windev

      @aallidina: Rename them and then put the symsrv.dll from the debugger folder there. That might make it work. If it doesn't, get a DBGHELP trace and email it to me (defragtools@microsoft,com)

      md c:\My
      md c:\My\DbgHelp

      setx DBGHELP_DBGOUT 1
      setx DBGHELP_LOG C:\My\DbgHelp\DbgHelpLog.txt

    • User profile image
      aallidina

      Strange. I finally got around to troubleshooting this, but can no longer reproduce.

      One possibility is that I had not restarted Visual Studio after converting the symbol store. Based on my Procmon trace, it appears that index2.txt is only queried on the very first symbol lookup (as opposed to flat.txt, which is queried all the time).

      I'll keep an eye on this, but it seems resolved. Thanks for your help.

    • User profile image
      Matt

      I've known about this for a while but my company has been running symbols stores since before I got to them we are looking at changing the store structure but haven't gotten around to it yet. We operate a symproxy server because we have multiple build machines each with there own symbol stores. However recently some of our developers have noticed windbg is making requests with the 2 tier system event though we haven't implemented it. Is there another setting within windbg that could be attempting to create this 2 tier directory even though we haven't got index2 in any of the caches?

    Comments closed

    Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.