Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

C9 Lectures: Mahmoud Saleh - Advanced CRT 1 of 2

Download

Right click “Save as…”

You first met Mahmoud Saleh in an episode of C9::GoingNative covering CRT (C Run-time Library). Mahmoud is the keeper of the CRT(C Run-time Library) at Microsoft, working on the VC++ team as a software engineer. The information presented in that GoingNative episode was introductory in nature and as we expected you asked for more advanced treatment of this important subject matter. Well, being the kind person that he is, Mahmoud agreed to do a full lecture covering advanced topics related to CRT. Thank you, Mahmoud! Smiley

Here, you'll learn about memory leak detection (CRT debug heap), unhandled exceptions, assert and error reporting, CRT entry points, CRT support for Unicode, SBCS and MBCC.

->Slides for this presentation

->Source code for the demos in this presentation


Tune in. Enjoy. Learn.

See part 2.

Tags:

Follow the Discussion

  • Sweet!  This sounds great!  Downloading.

     

  • I'm downloading it.

     

  • MattMatt

    Fascinating topic and great teacher, thank you Mahmoud!
    I really appreciated the technical depth of the talk.

    I have a few more questions and suggestions in case of a future episode ;)

    - Is it possible to write C programs that do not depend on the CRT? What are the limitations?

    - What is exactly the relationship between the Windows API and the CRT? It looks like the CRT is built on top of Win32 (e.g. malloc calls HeapAlloc) but I came across a few Win32 functions that actually rely on the CRT to do their work.

    - Is the Windows NT CRT dll (system32\msvcrt.dll) different from the one that ships with Visual Studio? Can we link our applications with this version of the CRT (and no longer ship the CRT dll with our code)?

    - I would love to hear more about the way SEH is implemented (what really happens at the cpu and memory level when an exception is thrown, the search for exception handlers, stack unwinding and so forth). The operating system and the compiler are definitely involved but is the CRT also a part of that?

    - I believe that some runtime floating point support is built inside the CRT but the source code is not available. Can you share with us some implementation details?

    - Ntdll.dll also contains some C runtime library functions (mostly string related though), are there similar to the ones implemented in the CRT?

    - How is the CRT designed to be thread-safe and do we still need to call functions like _beginthreadex instead of CreateThread to avoid memory leaks?

    Thank you!

  • BenBen

    Charles, Im so happy you're doing these.

    Keep up the good work :)

  • CharlesCharles Welcome Change

    @Ben: Thank you. Will do. Of course, Mahmoud did all the real work. Smiley So nice to have Mahmoud teaching us.

    More this of this type of thing to come (more STL/C++ lectures by STL, deep dive into C++/Cx design and implementation, and more...).

    C

  • BenBen

    Do you think we could get some interviews with the DirectX guys as well?

  • CharlesCharles Welcome Change

    @Ben: Yes. Do you mean VS11 DX11 tooling or DX itself or both?

    C

  • BenBen

    @Charles, I was thinking DX itself.

  • CharlesCharles Welcome Change

    @Ben: I'll see what I can do here!

    C

  • BenBen

    Awesome Sauce :)

  • C64C64

    Mahmoud and Charles: Thanks for the very interesting lecture.

    BTW: It seems to me that optimizing file I/O is not discussed... or am I missing something?

    It would be interesting to discuss also performance gain of raw Win32 API like CreateFile() with and without memory mapping vs. CRT fopen(). The latter has the advantage of being portable (and probably easier, too - e.g compare the long list of parameters of CreateFile() vs. the simpler interface of fopen()); but it'd be interesting to see if there are important advantages in using the raw Win32 API or if it is just a "wash".

    Thanks.

  • CharlesCharles Welcome Change

    @C64: Mahmoud, part 2? Smiley

    C

  • C64C64

    @Charles:Reading your description:

    "[...] Here, you'll learn about memory leak detection (CRT debug heap)unhandled exceptionsassert and error reportingCRT entry pointsCRT support for Unicode,SBCS and MBCCoptimizing file IO in CRT."

    I thought that optimizing file I/O was included in this lecture.

    Looking forward to part 2!  Smiley

  • new2STLnew2STL xkcd.com

    @C64: Yep,waiting for part2 too Big Smile. I think optimizing will deserve a full talk, as Mahmoud will be giving us hints on pagefile sizes (in the second last slide).

    This video reminded me that I should post some screenshots of debugging in my thread in the forums (about alignment for sse wrapped with std::unique_ptr).

  • CharlesCharles Welcome Change

    @C64: I based that on the slides. My fault! Smiley I will fix the description. I'll also see if Mahmoud is up for a part 2 (he's super busy, so I can't commit to a timeframe...).

    C

  • CharlesCharles Welcome Change

    @Charles: 1 of n Wink

    Thank you, Mahmoud!

    C

  • Can we have the usual MP4 versions please Charles.

  • CharlesCharles Welcome Change

    @dot_tom: Not sure what happened... Yes. I'll re-run the encoding/publishing job.

    C

  • evildictaitorevildictait​or Devil's advocate

     Matt said

    - Is it possible to write C programs that do not depend on the CRT? What are the limitations?

    Yes, but you have to choose not to. In Visual Studio you can go properties -> Linker -> Ignore All Default Libraries -> True. If you do this you'll also need to turn off buffer security checks and runtime checks in the compiler options and if writing a DLL in compiler -> Advanced set the entry point to "DllMain" or if writing an exe change it to "main".

    Things that have to live underneath the CRT (e.g. the Windows libraries, drivers and the kernel) need to do this if you want to build them with visual studio, but it's expected that most programs will want the CRT as it provides all sorts of nice low-performance overhead abstractions on top of the Windows functions exported by the Windows API.

    - What is exactly the relationship between the Windows API and the CRT? It looks like the CRT is built on top of Win32 (e.g. malloc calls HeapAlloc) but I came across a few Win32 functions that actually rely on the CRT to do their work.

    Microsoft's CRT is built on top of the Windows API (Win32). I'm not sure any core Win32 functions rely on the CRT, but do tell me if I'm wrong.



    - Is the Windows NT CRT dll (system32\msvcrt.dll) different from the one that ships with Visual Studio? Can we link our applications with this version of the CRT (and no longer ship the CRT dll with our code)?

    Yes. The Windows CRT is the one used by the Windows API dlls, e.g. kernel32.dll uses msvcrt.dll, but your dll will use the CRT shipped as part of your version of Visual Studio. The msvcrt.dll in system32 tends to change only with Windows Service Pack.

    - I would love to hear more about the way SEH is implemented (what really happens at the cpu and memory level when an exception is thrown, the search for exception handlers, stack unwinding and so forth). The operating system and the compiler are definitely involved but is the CRT also a part of that?

    In I386 processors (the x86 and x64 family if you want to call them that) use PAE to reference memory after Windows XP SP1. PAE allows pages of granularity 4096 bytes to be marked as "paged in" or "not available" and can put additional restrictions on them like "not executable", "not writable" or "uncacheable". When your CPU tries an invalid access to a page in memory, the CPU faults to the page fault handler in the kernel.

    A lot of the time page faults are "false positives" - you're touching some memory that is paged out (i.e. put in the pagefile to free up physical ram for other processes), part of a memory mapped file that hasn't been read yet and so on, and these page faults are serviced transparently by the kernel so that user-mode won't ever see them, but occasionally the page is marked as "not available" because you're trying to do something invalid like read an unmapped page or execute on a DEP-protected (NX) page.

    At this point the kernel SEH handler kicks in and tries to see if the previous mode was kernel mode. If it was, the kernel services the exception (or bluescreens). If the previous mode was user-mode, it queues an APC request to ntdll's KiUserExceptionDispatcher which is responsible for the user-mode part of SEH. ntdll checks if there are any vectored exception handlers that need to be run, then checks for SEH on the stack via the SEH exception chain. It calls each method in turn and if none of the registered exception handlers want to service the exception, ntdll tells the kernel to tear down the process and invoke the DrWatson crash dialog.

    The CRT is not typically involved in the SEH chain, but there are exceptions - the CRT does put SEH checks around several of its functions (i.e. the CRT uses SEH) and the compiler emits vectored exception handlers for many "SEH" try catch blocks (because vectored exception handlers are more safe from attack), but it's not really a CRT thing, it's more of an OS/ntdll thing.

    - I believe that some runtime floating point support is built inside the CRT but the source code is not available. Can you share with us some implementation details?

    The CRT does have code for "upgrading" the floating point performed on the CPU's FPU chip on processors that have low precision by essentially emulating the floating point behaviour in software. This is configurable in the properties of the compiler as either "fast" or "precise" floating point mode.

    - Ntdll.dll also contains some C runtime library functions (mostly string related though), are there similar to the ones implemented in the CRT?

    Yes and no. NTDLL is the ultimate base of the Windows user-mode - it's the first thing to be run in the process (long before your main function and long before the CRT) and it provides lots of functionality to the other Windows dlls such as kernel32, advapi, user32 and gdi32. The functionality that it provides is basically added on an ad-hoc basis almost like a somewhat manually invoked CRT for the Win32 libraries themselves, but it's not the same as the Visual Studio CRT even if they share code or if the CRT passes down to the NTDLL implementations.

    - How is the CRT designed to be thread-safe and do we still need to call functions like _beginthreadex instead of CreateThread to avoid memory leaks?

    Thank you!

    The CRT is mostly threadsafe, but you'll have to check on a per function basis. In particular the core functions like heap/thread/file/security are all thread safe.

    The _beginthreadex/CreateThread is an old issue, and what matters more than which to use is that you pair _beginthreadex's with _endthreadex and CreateThreads with ExitThread. _beginthreadex is better for the CRT (it gives you SEH wrappers for the entire thread for example) and mixing and matching will cause a small memory leak in the _beginthreadex/ExitThread direction and a possible crash in the CreateThread/_endthreadex direction, so choose one and stick to it.

  • CharlesCharles Welcome Change

    @Charles: Done. Thank you, Duncan!!
    C

  • MattMatt

    @evildictaitor Thank you for the thorough answers, much appreciated!

  • @Charles:Thanks Charles. I tend to watch these things on my A.p.p.l.e. device on the way to work. So a explicitly open and cross platform format is much appreciated.

  • CharlesCharles Welcome Change

    @dot_tom: We've supplied cross platform media encodings for a number of years for exactly this reason. Enjoy the show!

    C

  • Hi Mahmoud,

    Awesome lecture.

    When I tried to play with the sample code, I cannot step into CRT calls such as malloc() or free() in Debug mode, could you tell me how to enable this?

     

    Thannks

     

     

  • john sjohn s

    Is there a way to enable leak checking for 1 class only?

  • zofizofi

    plz send me such commented code.


    public static void Main(string[] args) {
    // Starts the application.
    Application.Run(new Form1());
    }

    private void button1_Click(object sender, System.EventArgs e) {
    // Populates a list box with three numbers.
    int i = 3;
    for(int j=1; j<=i; j++) {
    listBox1.Items.Add(j);
    }

    /* Determines whether the user wants to exit the application.
    * If not, adds another number to the list box. */
    while (MessageBox.Show("Exit application?", "", MessageBoxButtons.YesNo) ==
    DialogResult.No) {
    // Increments the counter ands add the number to the list box.
    i++;
    listBox1.Items.Add(i);
    }

    // The user wants to exit the application. Close everything down.
    Application.Exit();
    }

  • zofizofi

    how the user inpit is taken in c#?

  • zofizofi

    how the user input is takern in c#??

  • I've use the CRT leak detection for years, but lately our app needs to call waveOutGetNumDevs(), and as soon as you do this, leak detection breaks and no longer outputs memory leaks.

    I've documented it here:

    https://connect.microsoft.com/VisualStudio/feedback/details/658899/waveoutgetnumdevs-prevents-memory-leak-detection-output


    I've also seen instances where calling gethostbyname() breaks leaks detection as well, although I don't have a good reproducible test case yet.

  • Thank you, very interesting and helpful!

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.