Defrag Tools: #8 - Mark Russinovich

Play Defrag Tools: #8 - Mark Russinovich
Sign in to queue


Mark Russinovich joins Andrew Richards and Larry Larsen on this episode of Defrag Tools to talk about the history of Sysinternals, his involvement with the Windows Internals book series and advice on Cybersecurity. Learn about new tools, retired tools and tools that never got completed.  Get advice on troubleshooting. Get advice on how to survive a cyber attack. And much much more...

Write a comment before 24th Sept. for a chance to win a signed copy of Trojan Horse!

Mark's Blog (TechNet) -
Mark's Web Site -
Sysinternals Web Site -

All of Mark's videos on Channel 9 and talks at conferences. Of note:
* Case of the Unexplained...
* Mysteries of Memory Management Revealed - Part 1, Part 2
* Malware Hunting with the Sysinternals Tools
* RSA Conference 2012 -- Zero Day: A Non-Fiction View
* Inside Windows 7
* Inside Windows 7 Redux
* Windows 7 and Windows Server 2008 R2 Kernel Changes
* Windows Vista and Windows Server 2008 Kernel Changes

Sysinternals Administrator's Reference - [Amazon]
Windows Internals books:
* 4th Edition - Windows XP and Windows Server 2003 - [Amazon]
* 5th Edition - Windows Vista and Windows Server 2008 - [Amazon]
* 6th Edition - Windows 7 and Windows Server 2008 R2 - [Amazon: Part 1Part 2]
Cybersecurity novels:
* Zero Day - A Novel - [Amazon]
* Trojan Horse - A Novel - [Amazon]
* Operation Desolation - A Short Story - [Amazon]

[00:00] - How did Sysinternals start?
[02:20] - Tools that never got released and tool retirement
[03:55] - The most complex tool - Process Explorer
[04:51] - Favorite tool - ZoomIt
[07:01] - Windows Internals books
[10:54] - What's the best way to learn how to troubleshoot?
[12:47] - Do traditional techniques work when analyzing viruses?
[13:49] - Cybersecurity awareness
[14:40] - Cybersecurity novels
[16:28] - Cybersecurity advice for corporations and individuals
[20:25] - White Listing
[22:53] - User Account Control (UAC)
[29:55] - Winternals vs Sysinternals vs Windows Internals
[31:08] - New Windows 8 features/support in the Sysinternals tools:
Process Explorer v15.1
Process Monitor v3.0
* ProcDump v5.0
* RAMMap v1.2
DebugView v4.78
* AccessChk v5.1
[33:57] - Windows Internals 7th edition (for Windows 8)? Windows Azure Internals?
[36:47] - New tools - PsPing, RAMMap, VMMap
[40:33] - Win a signed copy of Trojan Horse!



Right click to download this episode

The Discussion

  • User profile image

    Fantastic work guys. Great that you could get Mark on.

    Love the show.

  • User profile image
    Jesse H

    Great information, love the channel! Awesome to see Mark on the show, he is a genius!

  • User profile image

    Thank you Mark,

    You expertise has improved the general security within the operating system and a great foundation for Azure. I am currently ready Zero Day and having been enjoying so much that my wife had to remind of the time and to turn off the light so she could sleep.


  • User profile image

    Mark your my hero man! Zero Day rocked. and Just started to read Trojan Horse. - Zero Day while fictional really open my eyes. I have spend the last 9 months learning Malware RE. I can honestly say your work inspire where I want my career to go. Hope to meet you one day.

  • User profile image

    Great show! Great talk and always new stuff to learn.

    I have Windows Internals 5 and I need another book to make a pair. And a signed one that would perfection! Need to order Zero Day and Trojan Horse, because in Portugal, book stores don't have it or is waiting to get it. It is hard to get really good books. Oh, Windows Internals 5... I waited like 4 months or so to get it. Anyway, I have to order both it and just wait.

    Also, this is an awesome comment. It is awesome cause it has the word awesome at least 3 times and it says comment too.


  • User profile image

    Mark, I like your shirt.

    Do I win?

  • User profile image

    Always love the talks from Russinovich, because he talks about the meat, the technology, the stuff under the hood for MS products, which a lot people seem to be quiet about. Also, cybersecurity - legit!

    The sheer knowledge you possess is just inspiring!


  • User profile image

    great stuff! i really liked the information about the history of sysinternals. thanks!

  • User profile image
    Mark Russinovich

    @Smoker65: lol!

  • User profile image

    This one is really superb ! Keep going !

  • User profile image

    "When in doubt run process monitor!"

    ... life is so much easier now Smiley

  • User profile image

    Oh now after watching the video I understand why "Prompt for elevation for non-Windows binaries" was introduced for Windows 7 UAC. But then why aren't all Windows binaries signed, at least why not some important ones like cmd or regedit?

  • User profile image

    Hey guys,

    I like how you go through all the sysinternals tools but how about some logging stuff? I'm a SharePoint (soon to be ex-) developer and becoming a SharePoint admin, so I embrace ULS Viewer Logging is helluva important here and it's actually the first thing I go to when someone tells me there's a problem.

    Do you know of any other log viewer/dig-througer (I tried logparse but it's kinda too rough for me) that can show me different logs like ULS, IIS, system, event viewer in real time with filtering, additional data (associated process information, correlation id, stack trace etc.) and stuff?

    I've also heard of some 'watson' log system, but it's kinda cryptic to me (only saw uls log entries like 'Error encountered, commencing Dr. Watson' or something). Is it relevant or ancient technology?

    Any hints on other useful logging toys?

  • User profile image

    @siodmy: We are going to do a big series on xPerf which will cover logging for all applications.  I'll add Logparser to the list of applications to be covered in a future episode.

  • User profile image
    Mark Russinovich

    @Gaurav: We didn't want to let ISVs easily cheat by leveraging cmd or regedit to modify the system for their apps with admin rights without a prompt. 

  • User profile image

    Big guns came out blazing today. Enjoyed the talk. Thanks.

  • User profile image

    Great video.  Thanks Mark and thanks Defrag team.

    I'm still chomping at the bit for part two of Win Internals 6th ed.  I have to admit, I felt a pang of sadness when you said you wouldn't be working on another edition of Windows Internals.  Not that anyone could blame you, as I know you're all about Azure now, and there's no doubt the Azure team is better for that.

    The 6th edition has been my first edition, and I felt like I got here late to the party, just as it was ending, as this book has been solid gold to me.  It's been exactly the kind of material that I soak up like a sponge.  I just really hope that someone can fill your shoes, pick up where you left off, and carry the torch of explicating the next generation of Windows Internals for the masses!

    That said, I'm also super excited to see what innovations Azure brings to the market. I'm a huge fan of cloud technologies, and they're keeping me employed right now, so I'm always looking for the newest and most exciting developments to come out of this industry.

    I also know that you will not stop writing tools.  Wherever you are, you'll keep writing tools to make whatever space you're in a better, more efficient, more informative, all around cooler place to be.

    After all, making tools is what really separates us from animals!

  • User profile image
    Court Oakes

    Does anyone remember the commercial Gatorade did about Michael Jordan??? "Sometimes I dream... that he is me... you know that's how I dream to be... like Mark... If I could be like Mark!" Seriously though, what he is doing, and has done, is analogous to what Jordan did with the game of basketball. He seems to be operating on a different plane. When I finished high school I really didn't have a much idea what I was going to do with my life. I worked for awhile, attended my local University for awhile, slowly working on a mathematics degree (I've always loved math) and as part of that I had to take a class in C++ programming. Well, while working on that, my brother mentioned that I should read about this genius that now works at Microsoft named Mark Russinovich. Well, I did, and it was then that I decided... that's what I want to do. Well, I am now a computer tech at a major retail outfit and am beginning my third year of study in Computer Engineering. I have read Zero Day (twice) and am half way through Trojan Horse and if you, like me, enjoy reading stories where you think to yourself, "this could really happen" then these books are for you. Anyway, I'd just like to take this opportunity to say thanks to Mark for being an incredible inspiration. Also, to say how cool it is that the public is finally beginning to understand the value of his work and to appreciate Mr. Russinovich not just as a computer scientist but as an engineer, a mathematician, an author, and as an all around artist.

  • User profile image

    @RyanRies: 6th edition Part 2 RTMed today, so it will be printed and available soon.

  • User profile image

    I really hope Zero Day and Trojan Horse are released as audiobooks at some point. But until then I want to win Trojan Horse Smiley

  • User profile image

    Also, it was mentioned that the UAC prompt doesn't show the cmd line but why not? Why is that single line hidden and user have to click "Show details" to view it every single time? Is there any way to always show details?

  • User profile image

    Thank you Mr Russinovich, I always have a dedicated monitor assigned to Process Explorer, even run it inside VM's and one day might get around to slip-streaming it into our Windows images as it's installed right after the first app 7-Zip. Process Monitor analysis should be forced labour for reformed hackers, though when you find the problem, you luckily forget the K's of lines and filters you've gone through. BUT! (oops caps-lock, apparently the visual studio design team also re-keyed that caps lock key!) Can we please have the Process Explorer graphs reset ( and better network graphs? Is the computer working? No, don't stare at the hdd light, look at the process explorer graphs!

  • User profile image

    Another great episode of defrag tools ... with legend of Mark R.

    I wolud like to share one "trojan" with you guys from my first flight ... and I hope that I'll get the real thing....real TROJAN HORSE Smiley) :

    A distinguished young woman on a flight from
    Croatia asked the priest beside her, "Father, may I ask a favor?"

    "Of course. What may I do for you?"

    "Well, I bought an expensive electronic hair dryer
    that is well over the Customs limits and I'm afraid they'll confiscate
    it. Is there anyway you could carry it through Customs for me?
    Under your robes perhaps?"

    "I would love to help you, dear, but I must warn you: I will not lie."

    "With your honest face, Father, no one will question you."

    When they got to Customs, she let the priest go ahead of her.
    The official asked, "Father, do you have anything to declare?"

    "From the top of my head down to my waist, I have nothing to declare."

    The official thought this answer strange, so asked, "And what do you
    have to declare from your waist to the floor?"

    "I have a marvelous little instrument designed to be
    used on a woman, but which is, to date, unused."

    Roaring with laughter, the official said, "Go ahead, Father. Next!"


    God bless defrag tools..... !!!

  • User profile image

    Thanks Mark and Channel 9 for the very interesting talk!

    And yes: Process Explorer is excellent!

    @Mark: It would be great if you could make Sysinternals tools open-source (e.g. sharing the source on CodePlex), so the community could both learn advanced Windows native programming techniques from your code and also contribute to code with additional features.

    Moreover, an analysis with depends.exe shows that Linker Ver field for procexp.exe is 9.0, meaning that Visual Studio 2008 (VC9) was used to build this tool. I'm curious why do you use this particular toolset (e.g. to support older OS'es like Windows 2000)?

    Thanks, and please keep up your excellent work on Sysinternals tools.

  • User profile image

    @C64: Visual Studio 2008 SP1 is used to compile the tools so that the tools use MSVCRT v9.0 - which is shipped with Windows XP/Windows 2003.

  • User profile image

    , windev wrote

    @C64: Visual Studio 2008 SP1 is used to compile the tools so that the tools use MSVCRT v9.0 - which is shipped with Windows XP/Windows 2003.

    I can be wrong, but using Dependency Walker I see no dependency of PROCEXP.EXE on MSVCR90.DLL, so I thought Sysinternals tools used static linking to CRT (which to me makes sense, to make tools deployment easier).


  • User profile image

    A very interesting Markinternals interview about the backstage of Sysinternals. An excellent wrap up of the series of Sysinternals Tools on C9. Unless there's more... Smiley

  • User profile image

    @StanS: There are a few more and then on to non-Mark tools.

  • User profile image

    As a software developer, I use PerfMon and ProcessExplorer a lot. Especially useful when trying to figure out when something doesn't work.

    Recently my team and I were trying to solve an issue with IIS AppPool because of high CPU usage. First think I thought of "Is there a tool which can take memory dump when these conditions occurs?". Then I checked Sysinternals and the tool was there, waiting for me. I somehow knew it will be there. Plus little bit of WinDbg, but that is different story Big Smile


    Thanks Mark for these great tools! They're making life a lot easier.

  • User profile image

    The more information your tools show, the less I know .. you know?

  • User profile image

    Mark, long time fan here, when we can expect the psping tool to be released. It would be of great use in network troubleshooting in organization I work for. We run VPN network layer on top of the WAN network topology which unfortunately hides a lot of the WAN network properties and makes performance planning and tunning hard (e.g. VPN layer makes the network hierarchy flat, in a way that the distance between all sites is always one hop, regardless of the physical network topology). I could run psping between endpoints in different sites to find the bottlenecks, it would help us a lot!

  • User profile image
    Mark Russinovich

    @bukem: It's great to hear you'll find psping useful. I'll be posting it in a couple of weeks. 

  • User profile image

    @Mark Russinovich:That's great news! And thank you for all the efforts you have made to keep the sysinternals tools up-to-date and moreover free.


  • User profile image

    @Mark Russinovich:BTW, it was nice to see you at TWIT finally!

  • User profile image

    Spoiler Alert:  don't read this if you haven't read the Zero Day book. 

    Mark, since the infection Jeff worked on was triggered by an incorrect date on the system, why couldn't he just reset the system with the correct date and then reinstall from backup?  Even if the backup was infected, it wouldn't be triggered until the trigger date (09/11).  Doing this would have allowed his client to get back up and running at least for a while. 

    Even if Jeff wasn't aware that the infection had been triggered by an incorrect date, when the system was rebuilt the first time, Sue (or even Jeff) should have set the rebuilt system to a correct date.  If the date was for some reason still wrong after the system was rebuilt, it should have raised a huge red flag and given them troubleshooting options. 

  • User profile image

    Spoiler Alert Part II - Don't read this if you haven't read the Zero Day book. 

    After figuring out that the infection had been triggered by an incorrect date, a quick workaround would have been to rebuild the system, set the date to a time after 09/11, and then restore the data from backup.  Obviously Time Stamp issues would be a concern, but at least the system would be up and running and the data would be accessible, etc.  That would give Jeff's client breathing room until a patch becomes available from the Vendors.  Does that seem technically sound for a quick workaround?  Or am I missing something? 


  • User profile image

    Wow- I can't believe that no one took this name before I did.

    Spoiler Alert Part III

    @Jamezs. It seems like your second scenario (Setting the date past 9/11) would work unless the trigger parameter was greater than or equal to 9/11.

    My questions are: Am I correct in assuming that the time settings are being provided to the client machines by the server(s). How could a company like Fischerman, Platt & Cohen not notice that the time settings were wrong on all of their workstations?

  • User profile image


    This DefragTools series is just ubercool (and hopefully never-ending) Smiley

    I've given all my co-workers an heads-up about this series, (and Mark's Case of the unexplained talks at TechEd, and other Sysinternals talks there), and their just amazed. There's tons of stuff to learn here. Some of us know the tools and use them, but some don't. Seeing them demonstrated by an experts is just 100 times better than just reading about them and trying by yourself.

    I hope You also can do a series focused on troubleshooting different scenarios, why You choose to use a specific tool, and how You use it. That's what so cool about the TechEd shows. It's a great way to learn the tools, and also the OS.  Especially an evolving one like Windows.  Can't get enough of that stuff...

    Hopefully Mark and You others on the team will continue posting bloggpost like the "Pushing the limits of Windows" series also. That one and talks like "Mysteries of Windows memory management" are packet with helpful insight into the inner workings of Windows.

    Don't stop, You're not finished... You're never finished. Go on... go on... go on...  Wink

  • User profile image
    Crispin Wright

    I just finished watching a talk about the payloads left to be decrypted inside flame, then I watched this one.....

    "Since each thread's TEB has its own Service Table List pointer, it is possible that every thread could also have its own unique table of OS services. However, in practice, the list and tables are globally shared. Simply changing an entry in either the NTOSKRNL or WIN32K service tables to point to a new hook routine in a device driver is all that is needed."

    I know someone else would have done it if you hadn't, but did you have any idea of the size of the pandoras box you were opening at the time?

  • User profile image

    Mark - loving your work! I was a Unix advocate until yours (& Bryse's) books and talks got me interested in the internals of Windows. The fact that Microsoft now employs you gives me renewed respect for the organisation.

  • User profile image
    Philip Churchill

    Trojan Horse, now thats a book I would like to win a copy off, and its signed by Mark too - awesome!

  • User profile image

    @DeepInsideTheDeathStar: Great series, great show! Keep it up guys! I´d really like to see a bit about malware hunting with the Sysinternals tools.

    @Mark: I really don´t know how you manage to keep all the balls in the air ... just astounding! "Zero Day" & "Trojan Horse" = movie material! Am still a little annoyed though that i can´t purchase "Operation Desolation" for my kindle. Still says "Not currently available" (seems that Amazon doesn´t like to sell in Germany?!?!)

    Cheers and all the best!

  • User profile image

    Rolling rolling rolling. Keep the books a flowin'. 

  • User profile image
    Rob Patterson

    Mark. I can't say enough good stuff about the sysinternal tools. They've been saving my sanity for years.

    Hey... If you're thinking about a fun new project (like you don't have enough on your plate), that Audiobook idea that Sailivi mentioned would be super cool. And I bet it would be ultra-awesome if you were the Narrator.  Cheers!

  • User profile image

    When in doubt, . . . run Process Explorer. . okay just kidding.

    However one legitimate question. . . Is there any chance that future Sysinternals tools could make their way into the PowerShell world?

    At a minimum tab completing params and inbuilt help and examples in the PowerShell standard format would be awesome.

    E.g. psping as a powershell cmdlet would be very sweet. I know old tools don't change much, however that is an example of a new one.

    As an ITPro I don't have an understanding about the effort required to make the transition, however I understand some of the benefits in discoverability of the tools and consuming the data returned.

  • User profile image

    I like the Sysinternals tools, my favorit is the Process Explorer Smiley
    Here are some questions about the tools.
    Why has the System Idle Process a Working Set and is counted in the sum of processes? Is there a real process behind?
    Is it possible to extend Process Explorer to show the app (process) history like the task manager in Windows 8? Is the history API public?
    Is it possible to extend the Process Dump tool to flush a ETW log in the case of a dump?

  • User profile image

    @SteffenZeidler: each core has a thread for idle processing. These are represented by PID 0 (which doesn't really exist). The threads consume working set as the threads need to be paged in to work.

    Process Explorer has history support. New history columns were added about a year ago. Instead of being numbers they are graphs. There is no explicit api that gives you the history. The closest thing is being an ETW consumer and polling the system with the tooltip32 API.

    ProcDump is designed to not change the state of the target. If you wrote your own MiniDumpCallback DLL (-d <dll>) you might be able to force the flush of the ETW buffers  - it'd only work if the target didn't needed to execute any of it's threads (as they will be all suspended).

  • User profile image

    So interetig to hear Mark talking... The systernal tools have helped thru the years and have made mike life easier

  • User profile image

    @windev: Thanks. By "app history" I mean the sum of resource usage of a process since a certain date.

  • User profile image

    I have never used the advanced tools of the SysInternals, however, simple tools such as ZoomIt, Autoruns and Autologon have made my work easier for many years. Thanks Mark!

  • User profile image

    I'd like a copy

  • User profile image

    @SteffenZeidler: Use the ETW and Tooltip32 APIs to get this data.

  • User profile image
    Mark Russinovich

    The winner of the show's signed book giveaway is - fittingly enough - Superphreak! @Superphreak, email your mailing address to and I'll send out the book. Congrats, Superphreak, and thanks everyone for the comments and feedback!

  • User profile image

    @Mark Russinovich, thanks for choosing my comment from among the other great posts on this page. It's an honor to receive a signed copy of Trojan Horse. I promise not to use its powers for 3vil. >Big Smile

  • User profile image

    I came recently with obscure bugs in Windows and ProcessExplorer hepled me to identify files caused explorer.exe to hang in one of its threads. I used handle.exe to indentify other known problem with COM Surrogate "Thumbnail Cache Out of Proc Server".
    Actually I think, Sysinternals tools saved a lot of Microsoft reputation!
    Thank you, Mark Russinovich, despite you won't read this comment :)

Add Your 2 Cents