Loading user information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading user information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements


Dexter Dexter
  • IL to C++ compiler - Need advice on implementing context switching

    Interestingly, the compiler gives me this warning:

    I'll answer this first because the rest can't be done properly with this in the way. The compiler can generate prolog/epilog code for a function:

                   push ebp         ; save the previous frame pointer
                   mov  ebp, esp    ; setup the frame pointer
                   sub  esp, 8      ; space for local variables
                   ...              ; function code
                   mov  esp, ebp    ; restore the stack pointer
                   pop  ebp         ; restore the frame pointer

    Now it should be obvious what's the warning about. The compiler uses ebp for its own purposes and warns you if you change it in inline asm. This compiler generated code can be a problem when you want to implement context switching because it affects the stack layout. You can implement SwitchContext even if such code is present but creating the stack for a new thread will be a problem because you don't know what stack layout SwitchContext expects.

    A possible solution is to use __declspec(naked) (which prevents such code from being generated) and __fastcall (which causes the first 2 arguments of the function to be passed in registers ecx and edx).

    I'm a bit rusty on my x86 assembly, but what I don't understand is that the code is moving oldStack into eax, then a few lines later, esp is moved into eax. Is my translation of the original code even correct?

    Nope. In the original version there were some parathesis which you ignored. Something like mov %esp, (%eax) converts to mov [eax], esp.

    Here's an example that uses __declspec(naked) and __fastcall):

    __declspec(naked) void __fastcall SwitchContext(VirtualThread* pOld, VirtualThread* pNew)
        // N.B. the following code assumes that NativeStack is at offset 4 in VirtualThread. If that's not true
        // then the appropiate offset needs to be used when storing/loading the stack.
            // Save registers
            push ebp
            push ebx
            push esi
            push edi
            // Save old stack
            mov [ecx+4], esp    // store to pOld->NativeStack
            // Load new stack
            mov esp, [edx+4]    // load from pNew->NativeStack
            // Restore registers
            pop edi
            pop esi
            pop ebx
            pop ebp

    So what I'm missing right now is a way to set up these stacks for the initial run

    To create a new thread you have to allocate memory for its native stack and setup the stack exactly the same as in SwitchContext does. Once the thread stack is properly initialized you can simply call SwitchContext to start the new thread.

    In addition, my assumption here is that my VirtualThread class will not only hold the "virtual stack", but also this native stack that will be used to do context switching for each thread.

    Yes, if you create thread you'll need to also allocate the native stacks yourself.

    void InitializeContext(VirtualThread* pNew, void *startFunction)
        int stackSize = 4096;
        char *stack = reinterpret_cast<char *>(malloc(stackSize)); // allocate some space, malloc is just an example
        stack += stackSize; // go to the top of the stack because the stack grows down
            mov ecx, esp // save current stack pointer
            mov esp, stack

            mov eax, startFunction
            push eax    // push the start function address as the return address (of SwitchContext)

            // Save registers
            xor eax, eax // let's always start a thread with zeroed registers
            push eax
            push eax
            push eax
            push eax

            mov stack, esp
            mov esp, ecx // restore the stack pointer
        pNew->NativeStack = stack;

  • IL to C++ compiler - Need advice on implementing context switching

    Closer inspection shows that as far as the test on line IL_0017 goes, it will always end up going to line IL_001d, hence it doesn't even bother.

    The test is not the problem, the problem is that the ldc 2 at IL_0019 is nowhere to be found in the generated C++ code. My guess is that the reason for this bug is the following:

    I'm using an evaluation stack based approach that accumulates expressions until an instruction comes along that does some real work, like storing

    The value (expression) that gets stored here was defined in two different places. Somehow you lost track of one of those definitions and ended up always writing 1. IL_001D can be reached with 2 different stack states but the generated code accounts for only for one.

    I'm just curious, what method is used to compile IL to native code?

    I don't know exactly what CLR does but what I'd do is convert the IL to some intermediate representation where there's no stack and where all instruction operands are explicit instead of implicit. Such explicit representation are far more easy to reason about.

    You may say that you aren't writing a compiler but getting familiar with some compiler techniques would be a good idea. Reading the code generation chapter(s) from the "Dragon Book" (http://dragonbook.stanford.edu/) is advisable.

  • Environment.​Set​Environment​Variable native c++ ?

    There must be one because otherwise Environement.SetEnvironmentVariable could not have been implemented Smiley




  • IL to C++ compiler - Need advice on implementing context switching

    If so, I need to get a new temp variable and assigne it the value of the variable, and also replace the previous variable with the temp variable everywhere it shows up on the stack.

    Mmm, seems like a cool solution to me.

  • IL to C++ compiler - Need advice on implementing context switching

     to step back through all of the previous instructions (up until the point it got pushed)

    Hmm, this is very easy to do when the push and pop are in the same basic block. If they're in different blocks then things can get complicate. One thing that you should note about these temporaries is that they have limited lifetime. Once a temporary is popped it can be reused to store another value.

    IMO, you're trying to get things working and optimize the code at the same time. I think it's better to just get things working first.

  • IL to C++ compiler - Need advice on implementing context switching

    I'm baffled. How did your compiler generate this line?

    IL_0041: // ldind.u2
    temp42 = ((UInt16)*(UInt16*)pStr);

    Since we talk that the stack contains values and not variables, how did pStr got in this statement? It should be temp38 instead.

  • IL to C++ compiler - Need advice on implementing context switching

    Hmm, you fixed the dup but the generated code is still not correct. In fact it behaves identically to the old code. It occurred to me now that the problem is more complicated than the buggy dup.

    The IL execution stack contains values. When you do a ldarg (or any other ld) the value stored in the variable is loaded on the stack. After a ld you can modify the variable in any way you like, the stack will still contain the original value of the variable.

    Your compiler treats the stack differently, it seems to store references to variables instead of values. This means that whenever you use a stack location that "references" a variable, you get the current value of the variable, not the original one that existed at the time ld was executed.

    Let's see this in the code:

    dupValue0 = pStr;
    pStr = (Char*)(((Int32)dupValue0 + (Int32)2));
    local0 = ((((UInt16)*(UInt16*)pStr) == 0) == 0);

    Despite it being different, its behavior is exactly the same. The last statement still uses the modified value of pStr instead of using the original one. The correct code should look like this:

    oldpStr = pStr;
    pStr = (Char*)(((Int32)pStr + (Int32)2));
    local0 = ((((UInt16)*(UInt16*)oldpStr) == 0) == 0);

    PS: editor problems: don't paste from VS, it seems that it generates html that confuses the post editor. Paste the code to notepad and then copy paste from notepad. This ensures that you paste plain text and not some weird html.

  • IL to C++ compiler - Need advice on implementing context switching

    Hmm, AFAIK "reach the dup instructions with different types" is not allowed. See Merging Stack States in ECMA 335.

    Even if this is allowed I think it's still possible to deal with. If a basic block containing dup is reached with 2 different stack states then this can be treated as if there actually were 2 distinct basic blocks in the first place.

  • IL to C++ compiler - Need advice on implementing context switching

    OK, let me detail this a bit:

    IL_003a: // ldarg.1 - load pStr
    IL_003b: // dup - duplicate pStr, now the stack contains the value of pStr twice
    IL_003e: // add - add one of the pStrs and 2
    IL_003f: // starg.s: Char* pStr - store the value on top of the stack to pStr
    // !!!the original value of pStr remains on top of the stack!!!
        pStr = (Char*)(((Int32)pStr + (Int32)2));IL_0041: // ldind.u2 - load from the address on top of the stack
    // that is, from the original pStr
        local0 = ((((UInt16)*(UInt16*)pStr) == 0) == 0); // load from pStr - not the original pStr but the incremented one

    The net result is that since you're using the incremented pStr you're one character ahead, this is where the -1 differences comes from.

  • IL to C++ compiler - Need advice on implementing context switching

    but it is going to bloat the code a lot (or at least the source files) since I'll need to create a unique structure for each call.

    Yes the sources will be a bit bigger but I'd say not by much. Note that you need one struct per method, not per call.