Tech Off Thread

21 posts

Forum Read Only

This forum has been made read only by the site admins. No new threads or comments can be added.

Unsafe C#

Back to Forum: Tech Off
  • User profile image
    DigitalDud

    It scares me that the C# compiler allows this:

    public byte* GetPointer()
    {
       fixed (byte* ptr = byteArray)
       {
          return ptr;
       }
    }

    This effectively returns the pointer to byteArray but leaves it unfixed by the runtime so it can still relocate byteArray whenever it wants. This will eventually leave your pointer pointing to some random memory area.

    I think it shouldn't be so easy to take a pointer out of the protection of a fixed block.

  • User profile image
    Cannot​Resolve​Symbol

    But what happens if you try to actually do something with the pointer outside of another unsafe block?  It may be something that throws a runtime exception if you try to use it rather than throwing a compiler error beforehand.

  • User profile image
    DigitalDud

    CannotResolveSymbol wrote:
    But what happens if you try to actually do something with the pointer outside of another unsafe block?  It may be something that throws a runtime exception if you try to use it rather than throwing a compiler error beforehand.


    Nope, there's no runtime exception. The pointer will actually work for awhile, until the garbage collector kicks in and decides to reorganize the heap.  Then the program will crash or do some whacky thing like a C++ program with a corrupted heap would do.

    See there's a performance penalty with using fixed blocks since the runtime has to be notified to "fix" the array in memory. I was hoping to avoid that with this code thinking it would leave the array perpetually fixed in memory. But instead it just slowly destroyed my program's heap. [C]

  • User profile image
    Foxfire

    Yes this can be a problem. Thats probably why it is called unsafe code and needs Full Trust to run and a special compiler-switch to compile Wink

  • User profile image
    Manip

    I would not like to see the compiler changed too much ... But think a warning about an unsafe pointer might be in order. I would welcome that.

    I accidentally created a large class that received and modified a tree of files in memory (ptr) but didn't enclose it in unsafe tags... The application always worked perfectly and still does. But if I modifed it down the road I might get random application corruption and it would be next to impossible to figure out way (that is assuming I hadn't already fixed it Smiley).

  • User profile image
    pacelvi

    Manip wrote:

    I would not like to see the compiler changed too much ... But think a warning about an unsafe pointer might be in order. I would welcome that.



    ROTFL.  Did you actually try it?

    Throw this in your friendly VS2005 IDE:

    using System;
    using System.Collections.Generic;
    using System.Text;
    namespace ConsoleApplication11 {
       class Program {
          static void Main(string[] args) {}
          public byte* GetPointer() {
             fixed (byte* ptr = byteArray) {
                return ptr;}}}}

    Here is what the compiler says:

    ------ Build started: Project: ConsoleApplication11, Configuration: Debug Any CPU ------

    C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Csc.exe /noconfig /nowarn:1701,1702 /errorreport:prompt /warn:4 /define:DEBUG;TRACE /reference:C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.Data.dll /reference:C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.dll /reference:C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.Xml.dll /debug+ /debug:full /optimize- /out:obj\Debug\ConsoleApplication11.exe /target:exe Program.cs Properties\AssemblyInfo.cs

    C:\Docs\Projects\ConsoleApplication11\ConsoleApplication11\Program.cs(13,16): error CS0214: Pointers and fixed size buffers may only be used in an unsafe context

    Compile complete -- 1 errors, 0 warnings

    ========== Build: 0 succeeded or up-to-date, 1 failed, 0 skipped ==========



    Manip wrote:


    I accidentally created a large class that received and modified a tree of files in memory (ptr) but didn't enclose it in unsafe tags... The application always worked perfectly and still does. But if I modifed it down the road I might get random application corruption and it would be next to impossible to figure out way (that is assuming I hadn't already fixed it ).

  • User profile image
    Manip

    It hasn't always reported it. I haven't tried it in 2005. I am however pleased that Microsoft has changed the compiler to warn the developer about accidentally doing so.

  • User profile image
    DigitalDud

    Manip wrote:
    It hasn't always reported it. I haven't tried it in 2005. I am however pleased that Microsoft has changed the compiler to warn the developer about accidentally doing so.


    I'm pretty sure you always needed an unsafe context to use pointers, unless there was a bug in the earlier version or something.

    Anyway, I read up the documentation on the fixed statement: http://msdn2.microsoft.com/en-US/library/f58wzh21(VS.80).aspx"> http://msdn2.microsoft.com/en-US/library/f58wzh21(VS.80).aspx

    The fixed statement sets a pointer to a managed variable and "pins" that variable during the execution of statement. Without fixed, pointers to movable managed variables would be of little use since garbage collection could relocate the variables unpredictably. The C# compiler only lets you assign a pointer to a managed variable in a fixed statement.

    The last line is incorrect since you can actually defeat the fixed statement by throwing it into a method and returning the pointer value, and you now have a pointer to a volatile block of memory.

  • User profile image
    AndyC

    DigitalDud wrote:


    The last line is incorrect since you can actually defeat the fixed statement by throwing it into a method and returning the pointer value, and you now have a pointer to a volatile block of memory.


    Yes, you can. This is "by design." It's one of the reasons that code which has an unsafe block is considered entirely unverifiable, because you can circumvent anything during that unsafe block. It's exactly why "unsafe" is unsafe.

    If you do this with your code, however, it will break. Even if it doesn't on your machine, it will on another. The aforementioned "performance" hit from using fixed is precisely because while the memory is pinned, garbage collection cannot operate optimally. Trying to circumvent that by leaving the fixed block before you are done will not help you, it'll just cause your code to crash in some random unpredictable way.

  • User profile image
    DigitalDud

    AndyC wrote:

    Yes, you can. This is "by design." It's one of the reasons that code which has an unsafe block is considered entirely unverifiable, because you can circumvent anything during that unsafe block. It's exactly why "unsafe" is unsafe.

    If you do this with your code, however, it will break. Even if it doesn't on your machine, it will on another. The aforementioned "performance" hit from using fixed is precisely because while the memory is pinned, garbage collection cannot operate optimally. Trying to circumvent that by leaving the fixed block before you are done will not help you, it'll just cause your code to crash in some random unpredictable way.



    It is by design but it's bad design.  Why not simply have a fix() and unfix() instead of a scoped block? With fixed blocks you end up with pretty looking code (that's sarcasm) like this:

    fixed (byte* ptrA = bufferA)
    {
       fixed (byte* ptrB = bufferB)
       {
          fixed (byte* ptrC = bufferC)
          {
             for (int i = 0; i < 100000; i++)
             {
                ptrA[i] = ptrB[i] + ptrC[i];
             }
          }
       }
    }

    Now obviously the point of having fixed be a block is to make sure the pointer will fall out of scope at the same time the managed object is unpinned. But given that these blocks are so easily defeatable, it would have made more sense to go with a more flexible object pinning ability that would allow you to have pointers to managed objects persist beyond a single scope.

  • User profile image
    Maurits

    Unsafe pointer manipulation is a feature.  It's a strong part of C's philosophy to give the programmer all the rope they want, even enough to hang themselves.

    In the Beginning was the Command Line -- Neal Stephenson

    I don't see what C++ has to do with keeping people from shooting themselves in the foot. C++ will happily load the gun, offer you a drink to steady your nerves, and help you aim.
        -- Peter da Silva

  • User profile image
    Sven Groot

    DigitalDud wrote:

    It is by design but it's bad design.  Why not simply have a fix() and unfix() instead of a scoped block? With fixed blocks you end up with pretty looking code (that's sarcasm) like this:

    fixed (byte* ptrA = bufferA)
    {
       fixed (byte* ptrB = bufferB)
       {
          fixed (byte* ptrC = bufferC)
          {
             for (int i = 0; i < 100000; i++)
             {
                ptrA[i] = ptrB[i] + ptrC[i];
             }
          }
       }
    }

    Actually, just like with using, you can do this with fixed blocks:

    fixed( byte* ptrA = bufferA )
    fixed( byte* ptrB = bufferB )
    fixed( byte* ptrC = bufferC )
    {
       for( int i = 0; i < 100000; i++ )
       {
          ptrA[i] = (byte)(ptrB[i] + ptrC[i]);
       }
    }

    That's a lot prettier don't you agree?

  • User profile image
    AndyC

    DigitalDud wrote:

    Now obviously the point of having fixed be a block is to make sure the pointer will fall out of scope at the same time the managed object is unpinned.



    No the point of having fixed be a block is that it forces the object to be unpinned. Pinning an object is very expensive and you really don't want to do it unless it's very necessary or for any longer than is absolutely required. It's not intended to protect you from doing anything stupid, if it was you wouldn't be in an unsafe block.

  • User profile image
    DigitalDud

    Sven Groot wrote:

    Actually, just like with using, you can do this with fixed blocks:

    fixed( byte* ptrA = bufferA )
    fixed( byte* ptrB = bufferB )
    fixed( byte* ptrC = bufferC )
    {
       for( int i = 0; i < 100000; i++ )
       {
          ptrA[i] = (byte)(ptrB[i] + ptrC[i]);
       }
    }

    That's a lot prettier don't you agree?


    That's pretty nice I'll give it a try. Smiley

  • User profile image
    DigitalDud

    AndyC wrote:


    No the point of having fixed be a block is that it forces the object to be unpinned. Pinning an object is very expensive and you really don't want to do it unless it's very necessary or for any longer than is absolutely required. It's not intended to protect you from doing anything stupid, if it was you wouldn't be in an unsafe block.



    In the context of my program there are no allocations made during normal execution. Therefore, there should be little pressure on the garbage collector to re-arrange the heap for much of the run-time.

    With memory usage patterns where all memory is allocated during initalization and not during normal usage, the price of pinning an object should be neglible. It's far more beneficial in such a program to have a fast persistent pointer access to an object than having to constantly pin and unpin the object.

    The fixed statement in C# cripples much of the functionality of pointers by not allowing such usage.

  • User profile image
    AndyC

    DigitalDud wrote:

    With memory usage patterns where all memory is allocated during initalization and not during normal usage, the price of pinning an object should be neglible. It's far more beneficial in such a program to have a fast persistent pointer access to an object than having to constantly pin and unpin the object.

    The fixed statement in C# cripples much of the functionality of pointers by not allowing such usage.


    Pinning and unpinning itself is not a particularly expensive operation, it's the overall effect of fragmenting the managed heap and hindering garbage collection that creates performance problems (this is why you are advised to allocate such buffers as early as possible, to accelerate their progress into the oldest generation of objects.)

    The overall effect of this is that in a situation such as you describe, the price will be negligable and you shoudn't notice any major impact from using fixed blocks. However, if worse comes to worse, you can always create pinned memory by using a GCHandle and passing GCHandle.Pinned to the Alloc method, a little dirty but it works.

    See Improving .NET Application Performance and Scalability chapter 5 for more details.

  • User profile image
    DigitalDud

    AndyC wrote:


    The overall effect of this is that in a situation such as you describe, the price will be negligable and you shoudn't notice any major impact from using fixed blocks. However, if worse comes to worse, you can always create pinned memory by using a GCHandle and passing GCHandle.Pinned to the Alloc method, a little dirty but it works.

    See Improving .NET Application Performance and Scalability chapter 5 for more details.


    Wow thanks for that. I'll read up on it.

    Anyway, this is all related to a personal emulator project that I'm working on.  It's been a constant battle against .NET in getting the performance up in what would have been a very fast C++ program, but I've learned a lot. I posted about it here: http://thecreamfilling.com/?ShowBlog=632459e7-08b6-4358-b277-73a48bc3b164

  • User profile image
    AndyC

    DigitalDud wrote:

    Anyway, this is all related to a personal emulator project that I'm working on.  It's been a constant battle against .NET in getting the performance up in what would have been a very fast C++ program, but I've learned a lot. I posted about it here: http://thecreamfilling.com/?ShowBlog=632459e7-08b6-4358-b277-73a48bc3b164


    Excellent, sounds like a really cool project.

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.