Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

martynl

martynl martynl

Niner since 2006

  • Inside Windows 7: Animation Manager Deep Dive and Tutorial

    This will be availble on Windows 7.  It will also be on Windows Vista via an update that should be available some time after Windows 7's general availability. A beta of the Windows Vista update will be available soon to MSDN subscribers.

     

    Martyn Lovell

    Development Lead

    Windows UI Platform

     

  • VC++: Safe Libraries with Martyn Lovell

    Sven,

    You're absolutely right, and this is why we have worked with the C standard committee on the C functions, and hope, based on experience gained from this relase, to work with the C++ committee on improving the safety of those libraries.

    For the default containers, algorithms and iterators, you end up being safer, performant and portable with little or no code change.

    However, if you start using custom algorithms or iterators, you end up with these choices, as the standard is currently. We support all three:

    You can be safer and portable, with performance loss in some cases. (usual constructs -- default is safe and portable)
    You can portable and performant, with a loss of safety (_SECURE_SCL=0)
    You can be safer and performant, with a portability loss (use _CHECKED_BASE, stdext::checked_array_iterator and so on).

    Though we can't always deliver all three simultaneously, we did work hard to make sure you could choose any point in the solution space you prefer, and to make sure that our default was both safe and portable as far as possible.

    Martyn

  • VC++: Safe Libraries with Martyn Lovell

    Sven,

    Your bubblesort example is very interesting. It illustrates exactly the challenges with having checked iterators and using general algorithms.

    Essentially, the checking inside the iterator can fire 'too often' in situations where algorithms work on understood bounds. Checking all boundaries once in a loop is overkill if you know that many initial conditions of the algorithm do not change.

    Turning off _SECURE_SCL isn't the right solution to this problem (though it's obviously a good short-term workaround). Instead, you can very easily create a both efficient and safe version of the algorithm.

    If you look at the code in our copy of <algorithm> you can see how this works - the algorithm code takes a checked iterator, does appropriate initial condition checking, and then gets an unchecked iterator from that checked iterator. It can then pass this unchecked iterator on to the core of the algorithm.

    If you need help, I'd be glad to work with you to build a more directly comparable version of the example.

    Martyn
  • VC++: Safe Libraries with Martyn Lovell

    Hi Leigh,

    You don't mention which version of VC or ATL you're using, but it sounds like you are somehow trying to target msvcrt.dll without using VC6. The only way to target msvcrt.dll is using VC6 (which is now out of support, and I strongly recommend against using).

    You should never copy a version of msvcrt.dll from a one version of the OS (in your example, Windows XP) to another. These OS-distributed msvcrt.dll are tightly bound to the OS in question, and must never be redistributed. They are also not licensed for redistribution, so you would be breaking your license if you did so.

    I think you'll find the value you get from msvcr80 (in terms of improved security and robustness) is worth the cost of redisting this DLL in all but the most bandwidth-constrained situations. I don't think it should stop you getting rocket-level performance Smiley

    > Here I have a problem on sizeof. the following foo shows szDen[]
    > and strDen[] both are the same array, why sz != sz1?

    Your question about sizeof illustrates one of the reasons we prefer _countof for array usage.

    > char* strcopy(char strDen[]/*char* strDen*/, const char* strSou)
    > {
    >      size_t sz1 = sizeof(strDen);  //sz1 always be 4?

    sizeof is a compile time language construct, so it can only tell you about the type information the language knows about during compilation. In the example you gave, strDen is a character array of unknown size, which in C is very similar to a char *. Your strcopy function can be called with any length of array, and C language semantics say that this kind of array doesn't come along with its size. So all that sizeof can tell you is the size of the pointer in question (which is obviously 4 on the CPU architecture you're targetting).

    Remember that sizeof can ultimately only tell you as much about a variable as the compiler knows during compilation. If you need more information than that, you'll have to add it as a parameter to the function.

    > And what’s the difference between strcpy_s and strncpy and
    > memcpy?

    memcpy: copies specified number of bytes without regard for their value. Just copies arbitrary data. Generally should not be used to copy strings.

    strncpy: copies up to 'n' characters from a NUL terminated string. If it finds a NUL in the source, it fills the rest of the 'n' with NUL. If it does not find a NUL, it fails to NUL-terminate the destination. Very hard to use this function safely and correctly. Theoretically good for truncation. In practice, strncpy_s with _TRUNCATE is much easier to use and known safe.

    strcpy_s: copies from source to a destination whose size is given. Attempts to copy whole of source. If whole of source can't be copied, empties contents of destination (NUL in position 0), and returns error/calls invalid parameter handler. Use strcpy_s when you know for sure that destination is big enough for source. If you don't know for sure, use strncpy_s with _TRUNCATE.

    Martyn


  • VC++: Safe Libraries with Martyn Lovell

    Slackmaster,

    Neat monitors Smiley I've never had more than 2 at once on one machine - both for desk space and angle reasons. Plus, my display card budget is limited... I'm looking forward to the improved multimon support in Remote Desktop in Windows Vista.

    Martyn

  • VC++: Safe Libraries with Martyn Lovell

    Sven,

    Thanks for following up.

    First, my reading of the standard is that the function at is required to do bounds checking, but operator at is also allowed to, because the behaviour of operator [] is said to be undefined outside of the valid ranges for the container. If I'm missing a standard paragraph, perhaps you can point me to it?

    Second, on behaviour when invalid situations occur.

    By default, when we find ourselves about to have an invalid condition, we call the operating system and ask it to collect a crash dump (this puts up the 'send error report' dialog or brings up a debugger). This can look like a "crash", but it is in fact a deliberate call to a specific OS API with known parameters designed to collect information about the fault. If you are an ISV, you can even sign up with Microsoft to get these crash dumps for your application, I believe.

    We recommend this approach because we generally think it's unsafe to continue running code in a process once that process is in an unknown state - lots of security exploits use such lax-ness to their advantage.

    However, we also know that developers have different preferences and opinions on what should happen. So you can use _set_invalid_parameter_handler to override the behaviour to cause something different to happen. And in the SCL, we added another option. If you'd like to get a SCL range exception when you hit this kind of condition, you can just define _SECURE_SCL_THROWS and then any of these range conditions will throw an exception.

    Your final point was about performance. I encourage you to post (or send me) your test code. Our experience was that most code recompiled with very little performance loss, and cases where there was perf loss were pretty easy to resolve. Because we put checking in every iterator operation, it's definitely possible to build 'worst-case' tiny examples where the cost looks very high. But for real world code I've worked with, the cost has mostly been pretty small, and certainly worth the improved safety. Of course, we provide _SECURE_SCL=0 for those cases where you really need all the perf and can do the error-checking yourself up front. But in our internal code we've rarely needed to use it.

    A key technique for making things perform well is to make sure you use std:: algorithm's to manipulate your container, since all of these have been perf optimised. If you write a manual for loop, the iterator has to check at each iteration. We are working on further optimisations for these cases in future versions, so any information you can give us about problems you're hitting would be very useful.

    I'd be glad to work with you to understand your perf situation in more detail. Similarly, if the error handling customisaion described above doesn't meet your needs, I'd be interested to understand that more.

    Thanks for the feedback, we appreciate it.

    Martyn
  • VC++: Safe Libraries with Martyn Lovell

    Hi Cairn,

    Good questions. Let me take the overloads point first.

    First, the 'automatic conversion feature'. The goal here is to reduce the cost for teams that want to migrate from existing risky code to safer code using our new functions. If you have code like this

    [all these code samples are typed live, so please excuse any errors]

    void foo(char *d, char *p)
    {
       // stuff

       strcpy(d,p);

       // ... stuff
    }

    then this is obviously risky. Our recommended mitigation changes the function to

    void foo(char *d, size_t ds, char *p)
    {
       // stuff

       strcpy_s(d, ds, p);

       // ... stuff
    }

    There's no way to do this kind of transformation automatically - the function calling convention had to change too.

    But for other kinds of function

    void foo(char *p)
    {
       char d[200];
       // stuff

       strcpy(d,p);

       // ... stuff
    }

    the mitigation is pretty trivial

    void foo(char *p)
    {
       char d[200];
       // stuff

       strcpy(d, _countof(d), p);

       // ... stuff
    }

    We noticed that if you are a C++ template programmer, you can write a 'clever' overload of strcpy that detects the use of a static array, and automatically works out the _countof the array and then calls strcpy_s. Defining

    _CRT_SECURE_CPP_OVERLOAD_STANDARD_NAMES

    in C++ code will engage this mapping. You can do similar mappings for functions taking counts (strncpy) with

    _CRT_SECURE_CPP_OVERLOAD_STANDARD_NAMES_COUNT

    Obviously, both these extensions provide template overloads of standard functions, and so aren't strictly conformant to the standard. So we turn them off by default. But you can turn them on when if you want them. We foudn that the significantly reduced the effort to secure existing code.

    There's no general way for us to correctly determine at compile time the intended buffer size of a buffer passed to strcpy, so we can't apply these techniques all the time. But when C++ is used, we can do some cases.

    You also mentioned the issue of the warnings using the word "deprecated", even though these functions are not deprecated in the relevant standard. We never intended this to imply deprecation in the standard sense, but it's caused a lot of concern and confusion, so in service pack 1 of our product we've removed the word 'deprecate' from the warnings associated with this, and replaced it with a text message that explains in more detail what we're doing here.

    Although the _s functions are not part of the C standard, we have worked very closely with the C committee on their design for the past 3 years, and a technical report of the C committee that includes many of them is under voting right now. I also know that a couple of other library implementers have portable implementations of these functions, so they should start turning up on other platforms. We wanted to try to help secure C code across the whole development community (not just Windows), which is why we worked so closely with the C committee on this issue.

    If you get a chance, take a  look at the latest C committee document and let me (or the editor) know what you think:
    http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1146.pdf

    Martyn

  • VC++: Safe Libraries with Martyn Lovell

    The first thing to say about _countof is that in C it has a simple definition ( sizeof(a)/sizeof(a[0]) ). But in C++ code it has a template magic definition which detects errors like

    char *p;
    _countof(p)

    which are a common source of buffer overruns in real code.

    From my perspective, this is enough reason to never use sizeof on arrays again. I've seen too many real issues result from it.

    If you build portable code, it's pretty easy to provide a trivial definition of _countof for other platforms. Even the template magic is portable, so you can do that too. There's no compiler magic here - you can check out the definition of _countof in the CRT headers.

    Martyn
  • VC++: Safe Libraries with Martyn Lovell

    Leigh,

    I wish I could give you a better answer to this question, but we don't have a good list of which ATL functions work right with ATL_MIN_CRT. The original intention of the ATL_MIN_CRT feature was that you could only use a 'minimal' subset of ATL that had no CRT dependencies, but this term 'minimal' was not strictly defined. As a result, as ATL has evolved, we've occasionally accidentally broken people with changes we've made. If you report these, we'll  try to fix them.

    Additionally, ATL has picked up new unavoidable CRT dependencies in some cases, where you now need the CRT.

    If you have specific cases you're seeing and need help with, do let me know

    Personally, I generally recommend against the use of ATL_MIN_CRT. It's still appropriate for a small number of situations where you're still dealing with very small redists, but for lots of cases it's an easier develop with a CRT DLL dependency. What's making you choose ATL_MIN_CRT?

    Martyn
  • VC++: Safe Libraries with Martyn Lovell

    Ion,

    That's a great question. We worked really hard to make the cost of these things at runtime in retail code very low. We've enabled them for our large codebases (VS, CLR, Office, SQL) without seeing major perf regressions.

    Generally, the overhead of checking and safety can be amortised so that it costs a lot less than doing the actual work of the function. You have to focus on calling a function lots of times to detect the cost of checking in general.

    For the Standard C++ Libraries, we put checking into iterators, which can have some significant perf cost in some cases (because iterators get used so often in a loop). But we made smart changes to our algorithms so that generally these costs are again amortised to once for the whole loop. As a result, std::for_each will be faster than for.

    We're working to further improve performance for future versions, but working with customers since our product released we haven't found a large number of perf problems, and have mostly been able to fix them with small tweaks.

    If you find a specific perf problem with our changes, please do get in touch with us.

    Martyn

See more comments…