• Do a static analysis of the code to determine which methods and types are actually in the code path, and ignore everything else. Done
• Create substitute types for all the commonly used types like Int8, Int16, Int32 etc, object, string, DateTime, TimeSpan, etc. Since a lot of the "native" types make internal calls, they need to be replaced with "pure" types. Unfortunately I would also need to create equivalents to other commonly used classes, since any methods that call into internal calls would be flagged as an error. Mostly done (although not all of the base type methods and properties have been implemented yet, just some to test the feasibility of the approach).
• Ensure the type system is fully managed. This means that the actual type of an "object" (or void*) can be determined at runtime. I can take this all the way to what reflection can do right now in .Net, it just depends on how much metadata I care to store in the exe. For instance, right now I don't store a type's namespace, its name or its member layouts etc, but it would be trivial to add it if I find I need it. Done.
• Implement generics. Done.
• Implement boxing/unboxing. Not done yet but should not be too difficult.
• Implement PInvoke. Not implemented yet but this should be relatively trivial since we are already in C++ land.
• Compile a C++ header file, declaring each used type and method. Done
• Compile a C++ source file with method implementations. Of course this involves the actual compiling from IL instructions to C++ code but from my previous experience with the IL interpreter I think this would be relatively easy although time consuming. Not done yet.

Advantages of this approach:

• You can take the C++ files and compile them for a microcontroller using any supported C++ compiler.
• This should run at almost "native" speed. Currently the .Net framework runs about 100 to 1000 times slower on a microcontroller than the equivalent native code.
• You can use the "full" .Net as opposed to just the .Net Micro Framework, although most of the framework classes' methods would not be available due to them making internal calls (this is a per-method limitation, so some methods might work just fine).
• The exe generated by this should be very small, since only the methods and types that are actually used in the call graph are compiled. For instance, if you never use the method DateTime.Now, that code will never make it into the final exe. This has a big advantage to even the Micro Framework, which still requires all of the .Net runtime to be present on the microcontroller.

I don't have a problem writing the part of the GC that walks the object graph marking all referenced objects and then deleting unreferenced ones. Since I have full control over the C++ class generation, I intend to implement something like this:

class VirtualObject
{
public:
    Bool m_mark;

    virtual void Mark()
    {
        m_mark = True;
    }
};

class VirtualMyClass : VirtualObject
{
public:
    VirtualObject m_variableA;
    SomeOtherObject m_variableB;

    virtual void Mark()
    {
        if (m_mark)
            return;   // prevent infinite loops if two objects reference each other directly or indirectly

        VirtualObject::Mark();

        if (m_variableA)
            m_variableA->Mark();

        if (m_variableB)
            m_variableB->Mark();
    }
};

The above code would be trivial to create. There would be a "virtual stack" where all "ref types" will be created and referenced to. Now all the GC has to do is to look at all the static variables and the "virtual stack", and call Mark() on all variables derived from VirtualObject that are not null. Then walk all of the currently allocated VirtualObject types and delete the ones that do not have the mark flag set, and clear the mark flag on the ones that do have it set (for the next pass).

OK, that was way more than I intended to write, but it helps to get the full picture of what I'm doing. Now to the questions I have...

So marking and deleting objects are taken care of. But what about compacting the remaining objects? I'm wondering how one would go about implementing VirtualObjects that are "movable". I have some ideas by they seem way too complex, and it involves having multiple levels of indirection. This doesn't seem ideal either because even if you have an array of objects and use an index as a "handle" for each, if you then allocate 1,000,000 objects, but only end up keeping a reference to the last one, you are stuck with a lookup array that has 999,999 items that are null but just the last one is not null. You can't compact the array since something is referencing to the item at index 999,999.

So what is the correct way to implement a system where object are fully "movable"? Even JITted code would still need to implement this. How do the smart guys do it?

Now all the GC has to do is to look at all the static variables and the "virtual stack", and call Mark() on all variables derived from VirtualObject that are not null.

This means that you know where the references to the objects are. If you know this then you can simply update the references when the objects move. An additional level of indirection is not required and it's not efficient anyway.

11 minutes ago,Dexter wrote

*snip*

This means that you know where the references to the objects are. If you know this then you can simply update the references when the objects move. An additional level of indirection is not required and it's not efficient anyway.

I think you are right. I was thinking that a currently executing function might have a local reference that could not be updated in such cases but if done correctly then this might not be a problem. So all "local" references to objects should really be references to objects on the "virtual object stack" as either a pointer to the location on the virtual stack, or an index to the virtual stack.

I'll need to give this some more thought but as I said I believe you are correct.

Somewhat related: Since microcontrollers don't support preemptive multithreading (or at least not the ARM microcontroller I'm playing with) I need to emulate that. The way I'm thinking of doing that is something like this:

void SomeClass::SomeFunction(VirtualThread* thread)
{
    // ...code...
    
    if (VirtualMachine.PendingContextSwitch)
        VirtualMachine.SwitchContext(thread);
    
    // ...code...
}

Usually the static VirtualMachine.PendingContextSwitch value would only be checked at the end of a function, or inside a loop (since there is no way to know how long the loop might take to complete). Also if the function is really long then I might add some additional checks.

The VirtualMachine.SwitchContext() call will do a context switch on the CPU. This is obviously very CPU dependent, and on the ARM processor you call the SWI instruction. Not sure how to do it on an x86 processor but it would not be too difficult to figure out (I'll stay away from x64 for now).

The VirtualMachine.PendingContextSwitch value itself would be set to true either periodically by a timer in VirtualMachine on x86, or by a hardware timer interrupt on the ARM processor.

The above mechanism has the advantage that I can control exactly where the code can be preempted, and maybe update all local references (if it is even needed) if VirtualMachine.SwitchContext returns true. Something like:

void SomeClass::SomeFunction(VirtualThread* thread)
{
    // ...code...

    if (VirtualMachine.PendingContextSwitch) && VirtualMachine.SwitchContext(thread)
    {
        // update all local references
    }

    // ...code...
}

Since most of the time PendingContextSwitch will be false, the check should not add much overhead, even inside tight loops. Once again, would this be the way the smart guys do it, or am I completely missing something here?

EDIT: Does anyone know what is up with the code formatting? Sometimes it insists putting everything on the same line, no matter whether you put newlines or Shift+newlines. I removed the code from the code blocks to make them readable.

This seems to be rather complicated to solve even in the absence of multithreading due to the fact that you generate C++ code, not assembly. Imagine this:

void SomeMethod() {
...
foo bar = new foo();
...
}

Let's say that new runs out of memory and triggers a GC. Let's also say that there's no threading and that the GC happens on the same thread. Even in this simple case you have no idea what the CPU registers contain when the GC code is entered, it's up to the compiler.

As for threading: I'm not familiar with these microcontrollers but I'm not sure why you would need all that machinery. As long as the CPU supports interrupts you should be able to implement preemptive multithreading without the need for this Pending/Switch stuff. And if the CPU doesn't have interrupts then who is going to set Pending anyway? You may need some form of cooperative multithreading in that case.

[Edit: I changed the code since there were some errors with the stack pointers]

class VirtualThread
{
    VirtualObject** StackBase;
    VirtualObject** StackPointer;
}
   

int SomeMethod(VirtualThread* thread)
{
    int retVal;
    VirtualObject** stackStart = thread->StackPointer;
    *thread->StackPointer = new foo();
    VirtualObject** bar = thread->StackPointer++;
    ((foo*)*bar)->DoSomething();
    // ...code...
   
    // The following lines of code would be changed to the code below it
    // if (someValue)
    //     return 123;

    if (someValue)
    {
        retVal = 123;
        goto Exit:
    }
   
    // ...code...

Exit:
    thread->StackPointer = stackStart;
    return retVal;
}

Or something along those lines. I do think there would need to be at least one level of indirection for local and static ref types.

Edit: I don't think with the above code there should be an issue with CPU registers since I am explicitly casting what is on the "virtual stack" to an instance of the actual object each time. The object can move between those calls. This is especially true if I use the context switching mechanism I proposed above.

Edit Edit: About the preemptive multitheading: If you think about it, even if you use a timer or interrupt, you still need to know which virtual thread to switch to. I'm not exactly sure about the details of context switching and register saving etc, but it will basically need to come down to saving all of the registers into some storage (preferably in the VirtualThread class), and then restoring the state from another VirtualThread and continuing execution where it left off.

So as far as the GC is concerned, now it just needs to enumerate through all VirtualThreads' stacks starting at StackBase up to StackPointer, and if it finds any non-null values there, it just calls VirtualObject::Mark() on it. Now you know exactly which objects are still being used as "locals" withing the executing functions (in addition to also calling VirtualObject::Mark() for each non-null static object).

Edit Edit Edit: I just realized that the "this" can also move while the class is executing code so it means that a "this" pointer will need to be passed on the stack as well, and all "this" references will need to go through this pointer as well.

Obviously this is getting more complex but I still think it would be way faster than the current performance we see with the Micro Framework.

I don't think with the above code there should be an issue with CPU registers since I am explicitly casting what is on the "virtual stack" to an instance of the actual object each time.

That doesn't matter too much. If you have code like the following:

((foo*)*bar)->DoSomething();((foo*)*bar)->DoSomething();

the compiler can (and usually will) perform the cast once and store the result into a register. However, this should work fine if you're allocating GC memory between those 2 lines. The compiler can't reuse the value from the previous cast because it has no way to prove that *bar wasn't changed by the memory allocation function.

Unfortunately this compiler optimization means that you have no way of doing normal preemptive multithreading, you have to use SwitchContext. I'm not what that will do to performance but I see no other way of doing this given the constraints.

I'm not exactly sure about the details of context switching and register saving etc, but it will basically need to come down to saving all of the registers into some storage (preferably in the VirtualThread class),

Pretty much yes. One possibility is to save the registers on the current thread's stack and then switch the stack. It's also possible that you won't need to save the registers if you use the SwitchContext approach because the compiler will probably save the "used" registers when it calls the function.

BTW I now make all functions static, even the instance functions, since I now also pass my own custom "this" pointer and there is no need for the real this pointer.

So, the following C# method...

class Foo
{
    int Multiply(int x, int y)
    {
        return x * y;
    }
}

...now becomes (at tleast the declaration)...

class Foo : public VirtualObject
{
    static int Multiply(VirtualThread* pThread, VirtualObject** pThis, int x, int y);
}

Edit: I can't make virtual functions static. I don't believe this will be a problem since AFAIC the virtual tables themselves are static, so it doesn't matter if the class instance moves around memory.

Surely the same issues apply?

Not really. Since CLR's JIT outputs assembly code it has knowledge about what's in the registers or about points where it is safe to stop the code for GC because there's no pointer in any register. Since you're generating C++ code you don't have this luxury, you're at the mercy of the compiler.

2 days ago,Dexter wrote

*snip*

Not really. Since CLR's JIT outputs assembly code it has knowledge about what's in the registers or about points where it is safe to stop the code for GC because there's no pointer in any register. Since you're generating C++ code you don't have this luxury, you're at the mercy of the compiler.

Actually the CLR doesn't make any assumptions like that. During a garbage collect on a particular thread, it suspends the thread (either by suspending it from another thread or during a call into the runtime, e.g. during a new object allocation). The registers are stored onto the stack and the stack is then inspected for any value that lies within the bounds of the current heap. Any value that falls within an object is marked as live for this round. The root-set computed is therefore a super-set of the "true" root-set, since an integer which happens to hold a value that would also be a valid pointer into the heap might cause a value to live longer than it strictly should, but this doesn't matter since this is much faster than using type information to derive the true root set.

At this point the thread can be resumed.

Added to the root set (which is rounded to the nearest object and now has type information which is stored with the object) is all of the objects from the other thread's root sets, all of the static variables (using type information) and thread local variables.

1) Mark all objects as "black"

2) Mark all objects from the current live set (computed above) as "gray"

3) While there are still objects in the gray set:

  4) Add all objects reachable from that object which are in the black set to the gray set.

  5) Add the object to the white set.

6) Goto 3 until the gray set is empty.

7) Now all objects reachable from the live set are colored white, and all objects colored black are not reachable from the live set.

2 days ago,BitFlipper wrote

Edit: I can't make virtual functions static. I don't believe this will be a problem since AFAIC the virtual tables themselves are static, so it doesn't matter if the class instance moves around memory.

Virtual tables aren't static:

class myClass {
  int a;
  static int b;
  virtual void myFunc();
  myClass() {}
}

looks like this in C:

static int _myClass_b;
typedef struct
{
  int m_a;
  void* myFunc
} MyClass; 

void _myClass(MyClass* cls)
{
  cls->myFunc = &MyClass_myFunc;
} 

Thanks for the info about the GC. I'm using a "virtual" stack that only contains ref types so I don't have the problem with objects being incorrectly flagged due to matching int values, etc.

As far as the vtables are concerned, I believe the vtables themselves are "static" in the sense that there is just one vtable for each type. So even if you create 100 instances of the same type, they all have a hidden pointer that point to the exact same vtable. Hence if you move the class around in memory, it won't matter since it will still point to the same vtable.

From Wikipedia: "Typically, the compiler creates a separate vtable for each class. When an object is created, a pointer to this vtable, called the virtual table pointer, vpointer or VPTR, is added as a hidden member of this object" http://en.wikipedia.org/wiki/Vtable

Actually the CLR doesn't make any assumptions like that.

Actually it does.

The registers are stored onto the stack

Actually no. While the kernel may indeed store the registers on the stack when the thread is suspended the only way for an application to get those registers is to call GetThreadContext. Sure, the CONTEXT that you pass to that function can be stored on stack but that's irrelevant here.

the stack is then inspected for any value that lies within the bounds of the current heap. Any value that falls within an object is marked as live for this round. The root-set computed is therefore a super-set of the "true" root-set, since an integer which happens to hold a value that would also be a valid pointer into the heap might cause a value to live longer than it strictly should

Nope, this is not what MS's CLR does. What you're describing here sounds like coservative GC, not compacting GC.

but this doesn't matter since this is much faster than using type information to derive the true root set.

Actually it matters a lot because it prevents objects from being moved which was one the main questions in this thread. You can't move objects unless you know exactly where pointers are. Or are you suggesting to modify random values in memory that happen to look like pointers?

Yes I'm finding a lot of complexities while trying to implement a compacting GC. The way I'm planning on doing it is with the "virtual stack" I mentioned previously. Each thread will have such a stack, and each of a function's ref type variables and parameters are really a pointer to a location on this stack like this:

MyRefType** foo = (MyRefType**)&(++pThread->StackPointer);

So while the value that is stored in the virtual stack can change as the class is moved around, foo itself never needs to change. When moving classes around, what I will need to do is check each value on each thread's virtual stack and update any pointers that have changed. Still trying to think what will be the most efficient way of doing this, but I believe in concept it will work.

For static ref type variables, I will just create one C function that is hardwired to check each class's static ref variables and call the Mark mentioned in the original post. Also there will be one C function that is hardwired to update any static ref pointers that have changed.

For class ref type variables, each class will have a hardwired function that can be used to update any of it's ref type pointers that could have changed. Something like:

void MyRefType::UpdatePointer(VirtualObject* oldPointer, VirtualObject* newPointer)
{
    if (m_foo == oldPointer)
        m_foo = newPointer;

    if (m_foo != NULL)
        m_foo->UpdatePointer(oldPointer, newPointer);
   
    // Do the same for all other class ref types...
}

[Sorry, I'm not doing the code blocks again, sometimes C9 loses it's mind and insists that all code should be on one line only. I'll just indent the text for now (which apparently doesn't seem to work either)]

Hmm, now that I think about it, that might not work because classes that are not reachable will never be called. If those classes are not rooted but they are still performing some sort of operations (maybe in some long running loop or a timer callback), their class variables will not be updated. So probably I will need to call UpdatePointer on each class while walking the object heap. In that case the UpdatePointer function will only update its own ref type variables, not call into UpdatePointer for each of its non-null ref objects. So it simplifies to:

void MyRefType::UpdatePointer(VirtualObject* oldPointer, VirtualObject* newPointer)
{
    if (m_foo == oldPointer)
        m_foo = newPointer;
    
    // Do the same for all other class ref types...
}

On a different topic, I ran into an interesting problem. So I have this concept that I will need to create "pure" classes due to the fact that many of the standard classes make internal calls, which I cannot decompile and create C++ code for. So just as an experiment, I added these lines of code to my C# test app:

var list = new List<int>();
list.Add(1);
list.Add(2);

And my C++ code suddenly got hugely bloated with all sorts of seemingly unrelated classes like the CultureInfo, etc with all of their huge amounts of static string resources etc. It seems that List is referencing those classes somewhere directly or indirectly, possibly through its static constructor. So I'm pulling in all of those types and doing a call graph analysis on their static constructors (which I need to call for all types I include), which is probably why it is detecting that all of those need to be included. Also in most cases it eventually hits an internal call and all bets are off.

Not sure how to work around that problem. Maybe I can do a better call graph analysis and only include static variables if the call graph actually references them (I might even do this for all variables - if they are never referenced in the call graph, I think they can be completely ignored, no?).

It won't be fun if I have to create "pure" versions of all the generic types, etc. But remember that the bar in this case is extremenly low: The .Net Micro Framework doesn't even support generics at all, so the fact that I do is already a big advantage given the ultimate goal of creating C++ code that can be compiled for a microcontroller.

Yet another different topic: Right now you can debug your .Net MF code running on a microcontroller using VS, just like any other .Net app (no edit and continue though). Since the .Net MF code is open source, I plan to figure out how it is done and ultimately (far down the line), I'd like to support debugging from VS. I can store as much or as little metadata for each type as I want, so I could make it such that using some VS plugin, I can map a current instance of any type back to the original type in the original .Net code (and pass it's member values to the debugger as well), so I think it should be possible to make a VS debugging plugin. Has anyone had experience with this, and how difficult would such a project be?

and each of a function's ref type variables and parameters are really a pointer to alocation on this stack like this:

One thing I don't understand is what are you going to do about value types that contain pointers and are stored on the stack. Did I miss something?

that might not work because classes that are not reachable will never be called. If those classes are not rooted but they are still performing some sort of operations (maybe in some long running loop or a timer callback), their class variables will not be updated.

That should never happen, if an object is not reachable then it's garbage and it will be collected. If you pass a GC pointer to native code then you know what you need to do: use "fixed" or GCHandle to prevent the GC from moving/collecting the object.

So probably I will need to call UpdatePointer on each class while walking the object heap.

Watch out that moving objects and updating pointers can be extremly tricky. I'm not talking about finding the roots which can be itself a problem, I'm talking about how can you update pointers in an efficient way. One thing is clear: you can't move one object and then call UpdatePointers on all the other objects in the heap, that has quadratic complexity.

Since evildictait​or mentioned it: have you considered using Boehm GC instead?http://en.wikipedia.org/wiki/Boehm_garbage_collector

And my C++ code suddenly got hugely bloated with all sorts of seemingly unrelated classes like the CultureInfo

Hmm, not sure what CultureInfo has to do with List<>. Maybe the Sort method uses it somehow. Since you wrote that code you should be able to modify it to log some information about why a particular type has been included, right? If I understand correctly now you're including all the methods of an used type, you may consider excluding methods that are never called.

so I think it should be possible to make a VS debugging plugin. Has anyone had experience with this, and how difficult would such a project be?

Absolutely no idea but if you can make all the rest work (compiler, GC etc.) then you should be able to write such a plugin too

1 hour ago,Dexter wrote

One thing I don't understand is what are you going to do about value types that contain pointers and are stored on the stack. Did I miss something?

Yes, a very good point. I've thought of two ways to solve that problem:

1. The "easy way out" would be to check whether a struct has any ref type fields and then treat it as a ref type (convert it to a class, which should be trivial). Not ideal since the point of using structs might be to put them in an array. But this could be the initial "solution".
2. A better but more difficult solution: Any structs that have ref type fields also get the same hard-coded functions to manage their pointers just like classes get (Mark and UpdatePointer). Then when the structs get initialized as locals in a function, a pointer to them is pushed onto the virtual stack. Now when the GC walks the stack, those structs will also have their pointer management functions called. For arrays, they can be treated as structs (not boxed), but the array class that holds structs (or ref types), will know how to call each of its array items' pointer management functions.

That should never happen, if an object is not reachable then it's garbage and it will be collected. If you pass a GC pointer to native code then you know what you need to do: use "fixed" or GCHandle to prevent the GC from moving/collecting the object.

Are you sure? Let's say the class has a method like this:

void MyClass.DoSomething()
{
    while (!someBoolThatWillAlwaysBeTrue)
        someRefObjField.DoSomethingElse();
}

That is certainly not "illegal" code and the compiler can't guard against it. Now if the original ref to MyClass goes out of scope, then it is no longer reachable and it will be GC'd (and hence someRefObjField won't be reachable either and will be GC'd).

Watch out that moving objects and updating pointers can be extremly tricky. I'm not talking about finding the roots which can be itself a problem, I'm talking about how can you update pointers in an efficient way. One thing is clear: you can't move one object and then call UpdatePointers on all the other objects in the heap, that has quadratic complexity.

Yes, and I was wondering about that too. Not sure if it would be faster to send a Dictionary<,> of changed pointers or not. I was thinking that I might just move one class at a time, hence making the GC "incremental". On a microcontroller it might be more important for the GC to not lock everything up for long periods of times instead of absolute performance. There is often somewhat of a "realtime" nature to microcontrollers with its hardware interrupts etc.

Since evildictait​or mentioned it: have you considered using Boehm GC instead?http://en.wikipedia.org/wiki/Boehm_garbage_collector

I'll take a look, thanks.

Hmm, not sure what CultureInfo has to do with List<>. Maybe the Sort method uses it somehow. Since you wrote that code you should be able to modify it to log some information about why a particular type has been included, right?

It possibly came in via one of the exception classes. I'll need to create substitutes for all of those too, so once I do, it might end up filtering out a lot of the unneeded stuff.

If I understand correctly now you're including all the methods of an used type, you may consider excluding methods that are never called.

That is what I'm already doing with my "call graph". I start at the Assembly.EntryPoint and look at all the IL instructions for any method calls. Any types that get referenced along the way gets added too. This process is repeated recursively for all methods found within the IL.

What I'm suggesting is improving on this process by stripping the fields out of types that were never referenced during all of the code paths. So if a class has a member field called SomeHugeClass, but the code path never references it, it can be safely removed it altogether. How else is it going to be referenced (other than via the static constructors which I'l treat in the same way)? Of course this makes reflection impossible but that is fine, I think reflection can be left out for now - it is a microcontroller after all.

A better but more difficult solution: Any structs that have ref type fields also get the same hard-coded functions to manage their pointers just like classes get

That sounds good. And I might have a slightly different idea, treat the stack frame as objects:

• for each method create a struct that contains all the method parameters and locals
• when you call a method allocate such a struct on the stack (by using alloca or other means)
• fill the parameters
• call the method with a pointer to the allocated struct
• you'll also need to store pointers to these stack frames on that virtual stack of yours or simply chain the stack frames into a linked list (pointed to by the thread)

Pros:

• "everything" is an object, you don't need separate cases for "real" objects and stack frames
• this works transparently with value types that contain references
• this might avoid the register problem completly. if a method makes a "alloc" call the compiler won't be able to prove that the stack frame object hasn't been modified and it will have to reload any registers that have been previously read from stack frame

Cons:

• This will very likely disable some compiler optimizations. Usually some of the parameters are passed through registers, this can't happen anymore since the only real parameter will be the pointer to stack frame. Similarly for local variables.

That is certainly not "illegal" code and the compiler can't guard against it. Now if the original ref to MyClass goes out of scope, then it is no longer reachable and it will be GC'd

MyClass can't go out of scope, if one of its methods is running then there must be a stack frame for it and that stack frame must contain the "this" pointer for MyClass.

On a microcontroller it might be more important for the GC to not lock everything up for long periods of times instead of absolute performance.

Sounds like you want generational GC too. Even more complicated .

What I'm suggesting is improving on this process by stripping the fields out of types that were never referenced during all of the code paths.

Sure, you could do that. Not sure if it helps with the List<T> case. I looked at the code and it's indeed possible that CultureInfo gets dragged in by an exception (see the Capacity prop). But it's unlikely that you can eliminate that by doing some static code analysis.

Your idea about the structure per function call is interesting but it is going to bloat the code a lot (or at least the source files) since I'll need to create a unique structure for each call.

My IL to C++ converter is coming along nicely. I have implemented roughly 70% of all IL instructions, enough to get a very good idea about performance. I tested it with a method that calls into another method with various parameters that then loads a string and does some basic math. It ran at about 80% - 90% of the C# version. It was able to complete ~71M calls/sec. Not too bad, considering I have not tried to optimize the code yet.

The following have been implemented:

• Generics
• Arrays
• Function calls (static, instance, virtual). P/Invoke should be easy to add.
• "Proxy" functions, used to handle static and instance calls on native types, like Int32, bool, etc. I chose not to create structs for these because I think it would be faster to just treat them as native types and use proxy functions (like Int32 Int32_Parse("123")).
• Strings. This includes the static (internal) strings, which are really fast to load (IL ldstr). I just pre-create an array of all of the static strings and specify the index to the array in the C++ code.
• The first part of the GC which includes making all ref types accessible in order to mark them before the sweep operations, and allowing updating of any pointers that have changed due to compacting. Classes are movable in theory, but I have not tested this yet.
• About 70% of all IL instructions.
• Some other stuff I forgot about.

I did run into a bizarre problem though, maybe someone can help me out.

I have the following C# method:

public class VirtualString : VirtualObject
{
    private UInt16 m_length;
    private unsafe char* m_buffer;

    public unsafe VirtualString(char* pStr)
        : base()
    {
        m_buffer = pStr;
        m_length = 0;

        if (m_buffer == null)
            return;

        while (*pStr++ != 0)
            m_length++;
    }
}

The IL looks like this (I modified ILSpy to show the instruction operand as well): 

    IL_0000: 0x02                    ldarg.0
    IL_0001: 0x28 0x060000a2         call instance void PostCompiler.VirtualTypes.VirtualObject::.ctor()
    IL_0006: 0x00                    nop
    IL_0007: 0x00                    nop
    IL_0008: 0x02                    ldarg.0
    IL_0009: 0x03                    ldarg.1
    IL_000a: 0x7d 0x040001c8         stfld char* PostCompiler.VirtualTypes.VirtualString::m_buffer
    IL_000f: 0x02                    ldarg.0
    IL_0010: 0x16                    ldc.i4.0
    IL_0011: 0x7d 0x040001c7         stfld uint16 PostCompiler.VirtualTypes.VirtualString::m_length
    IL_0016: 0x02                    ldarg.0
    IL_0017: 0x7b 0x040001c8         ldfld char* PostCompiler.VirtualTypes.VirtualString::m_buffer
    IL_001c: 0x16                    ldc.i4.0
    IL_001d: 0xe0                    conv.u
    IL_001e: 0xfe01                    ceq
    IL_0020: 0x16                    ldc.i4.0
    IL_0021: 0xfe01                    ceq
    IL_0023: 0x0a                    stloc.0
    IL_0024: 0x06                    ldloc.0
    IL_0025: 0x2d 0x02               brtrue.s IL_0029

    IL_0027: 0x2b 0x24               br.s IL_004d

    IL_0029: 0x2b 0x0f               br.s IL_003a
    // loop start (head: IL_003a)
        IL_002b: 0x02                    ldarg.0
        IL_002c: 0x25                    dup
        IL_002d: 0x7b 0x040001c7         ldfld uint16 PostCompiler.VirtualTypes.VirtualString::m_length
        IL_0032: 0x17                    ldc.i4.1
        IL_0033: 0x58                    add
        IL_0034: 0xd1                    conv.u2
        IL_0035: 0x7d 0x040001c7         stfld uint16 PostCompiler.VirtualTypes.VirtualString::m_length

        IL_003a: 0x03                    ldarg.1
        IL_003b: 0x25                    dup
        IL_003c: 0x18                    ldc.i4.2
        IL_003d: 0xd3                    conv.i
        IL_003e: 0x58                    add
        IL_003f: 0x10 0x01               starg.s pStr
        IL_0041: 0x49                    ldind.u2
        IL_0042: 0x16                    ldc.i4.0
        IL_0043: 0xfe01                    ceq
        IL_0045: 0x16                    ldc.i4.0
        IL_0046: 0xfe01                    ceq
        IL_0048: 0x0a                    stloc.0
        IL_0049: 0x06                    ldloc.0
        IL_004a: 0x2d 0xdf               brtrue.s IL_002b
    // end loop

    IL_004c: 0x00                    nop

    IL_004d: 0x2a                    ret
}

My IL -> C++ compiler creates the following C++ code: 

/////////////////////
// T8012_VirtualString.T8012_Ctor_VirtualString()
/////////////////////
Void T8012_VirtualString::T8012_Ctor_VirtualString(VirtualThread* pThread, T8012_VirtualString** pThis, Char* pStr)
{

    // Value type locals
    Bool local0 = false;

IL_0000: // ldarg.0
IL_0001: // call: Void .ctor()
    (*pThis)->T8003_VirtualObject::T8003_Ctor_VirtualObject(pThread, (T8003_VirtualObject**)pThis);

IL_0006: // nop
IL_0007: // nop
IL_0008: // ldarg.0
IL_0009: // ldarg.1
IL_000a: // stfld: Char* m_buffer
    (*pThis)->m_buffer = pStr;

IL_000f: // ldarg.0
IL_0010: // ldc.i4.0
IL_0011: // stfld: UInt16 m_length
    (*pThis)->m_length = 0;

IL_0016: // ldarg.0
IL_0017: // ldfld: Char* m_buffer
IL_001c: // ldc.i4.0
IL_001d: // conv.u
IL_001e: // ceq
IL_0020: // ldc.i4.0
IL_0021: // ceq
IL_0023: // stloc.0
    local0 = (((*pThis)->m_buffer == (Int32)(UInt32)(0)) == 0);

IL_0024: // ldloc.0
IL_0025: // brtrue.s: 2
    if (local0 != 0)
        goto IL_0029;

IL_0027: // br.s: 36
    goto IL_004d;

IL_0029: // br.s: 15
    goto IL_003a;

IL_002b: // ldarg.0
IL_002c: // dup
IL_002d: // ldfld: UInt16 m_length
IL_0032: // ldc.i4.1
IL_0033: // add
IL_0034: // conv.u2
IL_0035: // stfld: UInt16 m_length
    (*pThis)->m_length = (Int32)(UInt16)(((Int32)(*pThis)->m_length + 1));

IL_003a: // ldarg.1
IL_003b: // dup
IL_003c: // ldc.i4.2
IL_003d: // conv.i
IL_003e: // add
IL_003f: // starg.s: Char* pStr
    pStr = (Char*)(((Int32)pStr + (Int32)2));

IL_0041: // ldind.u2
IL_0042: // ldc.i4.0
IL_0043: // ceq
IL_0045: // ldc.i4.0
IL_0046: // ceq
IL_0048: // stloc.0
    local0 = ((((UInt16)*(UInt16*)pStr) == 0) == 0);

IL_0049: // ldloc.0
IL_004a: // brtrue.s: -33
    if (local0 != 0)
        goto IL_002b;

IL_004c: // nop
IL_004d: // ret
    return;
}

EDIT: Here is the C++ class declaration. Note that "Char" is defined as "wchar_t".

/////////////////////
// Original Name:  System.String
// Native Name:    T8012_VirtualString
// Native TypeRef: 0x8012
/////////////////////
class T8012_VirtualString : public T8003_VirtualObject
{
public:
    UInt16 m_length;
    Char* m_buffer;

    // Constructors
    T8012_VirtualString();
    Void T8012_Ctor_VirtualString(VirtualThread* pThread, T8012_VirtualString** pThis);
    Void T8012_Ctor_VirtualString(VirtualThread* pThread, T8012_VirtualString** pThis, Char* pStr);
    Void T8012_Ctor_VirtualString(VirtualThread* pThread, T8012_VirtualString** pThis, UInt16 length);

    // Special Functions
    virtual UInt16 GetTypeId() { return 0x8012; }
    virtual Bool IsInstanceOf(UInt16 otherId) { return otherId == 0x8012 || otherId == 0x8003; }

    static Int32 get_Length(VirtualThread* pThread, T8012_VirtualString** pThis);
};

The problem is that the code does something different from what the original one does. It always sets m_length to one less than what it should be. The C# version works correctly (pass in "123", and m_length is set to 3), but the C++ version leaves m_length one too small. I'm confused as to how that can happen (pass in "123" and m_length is set to 2). As far as I can see, my C++ code does exactly what the IL tells it to do. Any ideas?

Note I can turn the IL labels and instruction comments on/off but turned both on to see the relationship to the original IL better.

EDIT: OK I found that if I change the C# code to this:

        public unsafe VirtualString(char* pStr)
            : base()
        {
            m_buffer = pStr;
            m_length = 0;

            if (m_buffer == null)
                return;

            while (*pStr != 0)
            {
                m_length++;
                pStr++;
            }
        }

I get the right results in the C++ version. This indicates that the problem is in how I handle "*pStr++".

EDIT EDIT:I wonder if it has something to do the the "dup" instructions. The way I handle dup is to clone the value of the item on the top of the "virtual" stack and push it so that there are two. Also, the way this virtual stack works is that it contains text that represents the local variable, parameter name or a previous operation. So in the case of the dup at IL_003b, the text on the stack will be "pStr". So when I perform the dup operation, the two top items on the stack now contain "pStr". I wonder whether I need to create a completely new local variable when I come across a dup operation.

I wonder if it has something to do the the "dup" instructions.

Yep. If you look at the generated code for IL3A-3F you can easily note that dup has no effect. If you remove the dup then your compiler would generate identical code. That's a good indication that something isn't right.

So when I perform the dup operation, the two top items on the stack now contain "pStr".

dup make a copy of the value, it seems to me that your implementation makes a copy of some sort of reference to the value.

but it is going to bloat the code a lot (or at least the source files) since I'll need to create a unique structure for each call.

Yes the sources will be a bit bigger but I'd say not by much. Note that you need one struct per method, not per call.

12 minutes ago,Dexter wrote

*snip*

Yep. If you look at the generated code for IL3A-3F you can easily note that dup has no effect. If you remove the dup then your compiler would generate identical code. That's a good indication that something isn't right.

*snip*

dup make a copy of the value, it seems to me that your implementation makes a copy of some sort of reference to the value.

I'm trying to understand how it is failing but I can't. To me, those instructions want to add 2 to the value of pStr, which it clearly does via this line of C++ code:

pStr = (Char*)(((Int32)pStr + (Int32)2));

Also, stepping through the code, the problem is not that pStr is incremented wrong, it is that m_length is incremented too late. You might be right but I can't figure it out.

Note the way the IL instructions are notated in the C++ code is that all IL instructions leading up to actual code are grouped together. So all instructions from IL3A-3F results in the line of code shown above. From what I can tell it is performing the correct operation on pStr. pStr showing up twice in the line of code is a direct result of the dup operation.

I think the bug must be somewhere else.

IL_003a: // ldarg.1 - load pStr
IL_003b: // dup - duplicate pStr, now the stack contains the value of pStr twice
...
IL_003e: // add - add one of the pStrs and 2
IL_003f: // starg.s: Char* pStr - store the value on top of the stack to pStr
// !!!the original value of pStr remains on top of the stack!!!
    pStr = (Char*)(((Int32)pStr + (Int32)2));IL_0041: // ldind.u2 - load from the address on top of the stack
// that is, from the original pStr
...
    local0 = ((((UInt16)*(UInt16*)pStr) == 0) == 0); // load from pStr - not the original pStr but the incremented one

The net result is that since you're using the incremented pStr you're one character ahead, this is where the -1 differences comes from.

Yes, I think you are correct. I'll need to figure out how to create temporary variables in the case of dup. Not as straightforward since ref types need to be stored on the virtual stack to make sure they are not GCd, and those are prepared when the function enters.

We have company coming over so I won't be able to look at this again until tomorrow or so.

EDIT: Actually, this might not be so difficult. I already do a pre-pass and attach local variables to certain instructions (newObj, newArr, call, etc), so I can do the same for dup. Then when the function code is written, I check all instructions for local variables (either ref of value types), and create that part of the code. So all I need to do is also create a local variable for dup to store its duplicated value in.

So what kind of local variable needs to be created ahead of time to store the duplicate value? An object type? Then if the object on the stack is a value type, the value needs to be boxed.

Another option could be to determine all possible locations that could jump to the dup and determine what would be on the stack at that time. At runtime, choose the reserved local variable of the right type. Seems complicated but then again it probably would not be too common to find a dup where multiple areas of the code jumps to it.

Is there a better solution?

public unsafe VirtualString(char* pStr)
{
    this.m_buffer = pStr;
    this.m_length = 0;
    if (this.m_buffer != null)
    {
        while (true)
        {
            char* expr_3B = pStr;
            pStr = expr_3B + (IntPtr)2 / 2;
            if (*(ushort*)expr_3B == 0)
            {
                break;
            }
            this.m_length += 1;
        }
    }
}

So that tells me that it clearly sees the temporary variable as the same type as pStr, and hardcoded as such. I assume that in this case it would be "illegal" IL if you did have a situation where you can reach the dup instruction with different types on the top of the stack. The IL documentation doesn't mention this detail at all. Neither does the book "The Common Language Infrastructure Annotated Standard". It literally just says "The dup instruction duplicates the top element of the stack". Nothing else.

Even if this is allowed I think it's still possible to deal with. If a basic block containing dup is reached with 2 different stack states then this can be treated as if there actually were 2 distinct basic blocks in the first place.

6 hours ago,Dexter wrote

Hmm, AFAIK "reach the dup instructions with different types" is not allowed. See 1.8.1.3 Merging Stack States in ECMA 335.

Even if this is allowed I think it's still possible to deal with. If a basic block containing dup is reached with 2 different stack states then this can be treated as if there actually were 2 distinct basic blocks in the first place.

I looked at section 1.8.1.3 and it mentions how types can be converted. This does help me completely because during my code analysis lets say I find an Int16 on the stack. Now I create a local variable of type Int16. But the at runtime something jumps to the dup instruction from another part of the code, and there happens to be an Int32 on the stack. While I can cast to an Int16, I'm going to lose precision. For now I'm going to go with the assumption that there will always be the same type on the stack when reaching a dup instruction.

Anyway, I went ahead and changed the code to create an actual local variable for each dup instruction (with the assumption that it can always only be of the type that was on the stack during the initial code analysis pass). Unfortunately I am still seeing the bug where m_length is 1 less than it should be. Below is the new generated C++ code. To me it seems that instruction IL_0029 should jump to instruction IL_002d instead of IL_003a. It is basically skipping the first increment on m_length. Can you spot the bug in my C++ code?

[EDIT: OK, I'm currently struggling with C9's code formatter in order to not have it put all of the code on one line. I'm selecting the code in VS, copying it, opening the "</>" code editor dialog in C9, pasting it, selecting the language type, pressing Insert and then Submit. What am I doing wrong? For now, I'm pasting the unformatted code in addition to the single formatted line until I can figure the formatting out]

        Void T8012_VirtualString::T8012_Ctor_VirtualString(VirtualThread* pThread, T8012_VirtualString** pThis, Char* pStr)
        {

            // Ref type locals
            T8012_VirtualString** dupRef0 = (T8012_VirtualString**)&pThread->StackPointer[1];

            memset(&pThread->StackPointer[1], 0, sizeof(void*) * 1);
            pThread->StackPointer += 1;

            // Value type locals
            Bool local0 = false;
            Char* dupValue0 = 0;

        IL_0000: // ldarg.0
        IL_0001: // call: Void .ctor()
            (*pThis)->T8003_VirtualObject::T8003_Ctor_VirtualObject(pThread, (T8003_VirtualObject**)pThis);

        IL_0006: // nop
        IL_0007: // nop
        IL_0008: // ldarg.0
        IL_0009: // ldarg.1
        IL_000a: // stfld: Char* m_buffer
            (*pThis)->m_buffer = pStr;

        IL_000f: // ldarg.0
        IL_0010: // ldc.i4.0
        IL_0011: // stfld: UInt16 m_length
            (*pThis)->m_length = 0;

        IL_0016: // ldarg.0
        IL_0017: // ldfld: Char* m_buffer
        IL_001c: // ldc.i4.0
        IL_001d: // conv.u
        IL_001e: // ceq
        IL_0020: // ldc.i4.0
        IL_0021: // ceq
        IL_0023: // stloc.0
            local0 = (((*pThis)->m_buffer == (Int32)(UInt32)(0)) == 0);

        IL_0024: // ldloc.0
        IL_0025: // brtrue.s: 2
            if (local0 != 0)
                goto IL_0029;

        IL_0027: // br.s: 36
            goto IL_004d;

        IL_0029: // br.s: 15
            goto IL_003a;

        IL_002b: // ldarg.0
        IL_002c: // dup
            (*dupRef0) = *(pThis);

        IL_002d: // ldfld: UInt16 m_length
        IL_0032: // ldc.i4.1
        IL_0033: // add
        IL_0034: // conv.u2
        IL_0035: // stfld: UInt16 m_length
            (*pThis)->m_length = (Int32)(UInt16)(((Int32)(*dupRef0)->m_length + 1));

        IL_003a: // ldarg.1
        IL_003b: // dup
            dupValue0 = pStr;

        IL_003c: // ldc.i4.2
        IL_003d: // conv.i
        IL_003e: // add
        IL_003f: // starg.s: Char* pStr
            pStr = (Char*)(((Int32)dupValue0 + (Int32)2));

        IL_0041: // ldind.u2
        IL_0042: // ldc.i4.0
        IL_0043: // ceq
        IL_0045: // ldc.i4.0
        IL_0046: // ceq
        IL_0048: // stloc.0
            local0 = ((((UInt16)*(UInt16*)pStr) == 0) == 0);

        IL_0049: // ldloc.0
        IL_004a: // brtrue.s: -33
            if (local0 != 0)
                goto IL_002b;

        IL_004c: // nop
        IL_004d: // ret
            goto Exit;

        Exit:
            pThread->StackPointer -= 1;
        }

[Does the following block of code display properly for anyone? It is all on one line for me. I filed a C9 bug report about this and referred back to this post. I'll leave this extra block of code here until the bug in the forum has been fixed.]

void T8012_VirtualString::T8012_Ctor_VirtualString(VirtualThread* pThread, T8012_VirtualString** pThis, Char* pStr){    // Ref type locals    T8012_VirtualString** dupRef0 = (T8012_VirtualString**)&pThread->StackPointer[1];    memset(&pThread->StackPointer[1], 0, sizeof(void*) * 1);    pThread->StackPointer += 1;    // Value type locals    Bool local0 = false;    Char* dupValue0 = 0;IL_0000: // ldarg.0IL_0001: // call: Void .ctor()    (*pThis)->T8003_VirtualObject::T8003_Ctor_VirtualObject(pThread, (T8003_VirtualObject**)pThis);IL_0006: // nopIL_0007: // nopIL_0008: // ldarg.0IL_0009: // ldarg.1IL_000a: // stfld: Char* m_buffer    (*pThis)->m_buffer = pStr;IL_000f: // ldarg.0IL_0010: // ldc.i4.0IL_0011: // stfld: UInt16 m_length    (*pThis)->m_length = 0;IL_0016: // ldarg.0IL_0017: // ldfld: Char* m_bufferIL_001c: // ldc.i4.0IL_001d: // conv.uIL_001e: // ceqIL_0020: // ldc.i4.0IL_0021: // ceqIL_0023: // stloc.0    local0 = (((*pThis)->m_buffer == (Int32)(UInt32)(0)) == 0);IL_0024: // ldloc.0IL_0025: // brtrue.s: 2    if (local0 != 0)        goto IL_0029;IL_0027: // br.s: 36    goto IL_004d;IL_0029: // br.s: 15    goto IL_003a;IL_002b: // ldarg.0IL_002c: // dup    (*dupRef0) = *(pThis);IL_002d: // ldfld: UInt16 m_lengthIL_0032: // ldc.i4.1IL_0033: // addIL_0034: // conv.u2IL_0035: // stfld: UInt16 m_length    (*pThis)->m_length = (Int32)(UInt16)(((Int32)(*dupRef0)->m_length + 1));IL_003a: // ldarg.1IL_003b: // dup    dupValue0 = pStr;IL_003c: // ldc.i4.2IL_003d: // conv.iIL_003e: // addIL_003f: // starg.s: Char* pStr    pStr = (Char*)(((Int32)dupValue0 + (Int32)2));IL_0041: // ldind.u2IL_0042: // ldc.i4.0IL_0043: // ceqIL_0045: // ldc.i4.0IL_0046: // ceqIL_0048: // stloc.0    local0 = ((((UInt16)*(UInt16*)pStr) == 0) == 0);IL_0049: // ldloc.0IL_004a: // brtrue.s: -33    if (local0 != 0)        goto IL_002b;IL_004c: // nopIL_004d: // ret    goto Exit;Exit:    pThread->StackPointer -= 1;}

The IL execution stack contains values. When you do a ldarg (or any other ld) the value stored in the variable is loaded on the stack. After a ld you can modify the variable in any way you like, the stack will still contain the original value of the variable.

Your compiler treats the stack differently, it seems to store references to variables instead of values. This means that whenever you use a stack location that "references" a variable, you get the current value of the variable, not the original one that existed at the time ld was executed.

Let's see this in the code:

dupValue0 = pStr;
pStr = (Char*)(((Int32)dupValue0 + (Int32)2));
local0 = ((((UInt16)*(UInt16*)pStr) == 0) == 0);

Despite it being different, its behavior is exactly the same. The last statement still uses the modified value of pStr instead of using the original one. The correct code should look like this:

oldpStr = pStr;
pStr = (Char*)(((Int32)pStr + (Int32)2));
local0 = ((((UInt16)*(UInt16*)oldpStr) == 0) == 0);

PS: editor problems: don't paste from VS, it seems that it generates html that confuses the post editor. Paste the code to notepad and then copy paste from notepad. This ensures that you paste plain text and not some weird html.

So basically for any instruction that pushes either a local or parameter value, I need to create a temporary local variable.

Hopefully the compiler can optimize most of those away, although it won't be able to do it for ref types which I store on the virtual stack.

An optimization in this case could be to keep track of the temp locals during the analysis pass and mark them as "free" when they get popped. Then they can be re-used the next time a local or parameter needs to be pushed on the stack (if they are of the same type of course). Basically a pool of temp locals.

About the code formatting: I tried everything, including pasting into Notepad first. It simply refuses to format the code correctly. I have had success with both pasting directly from VS (but into the popup "</>" code dialog), or via pasting dirently into the text editor after pasting into Notepad. I think the problem in this case is that a previous formatting tag somewhere on the page is confusing the formatter.

Void T8013_VirtualString::T8013_Ctor_VirtualString(VirtualThread* pThread, T8013_VirtualString** pThis, Char* pStr)
{

    // Ref type locals
    T8013_VirtualString** temp0 = (T8013_VirtualString**)&pThread->StackPointer[1];
    T8013_VirtualString** temp1 = (T8013_VirtualString**)&pThread->StackPointer[2];
    T8013_VirtualString** temp3 = (T8013_VirtualString**)&pThread->StackPointer[3];
    T8013_VirtualString** temp4 = (T8013_VirtualString**)&pThread->StackPointer[4];
    T8013_VirtualString** temp10 = (T8013_VirtualString**)&pThread->StackPointer[5];
    T8013_VirtualString** temp11 = (T8013_VirtualString**)&pThread->StackPointer[6];
    T8013_VirtualString** temp23 = (T8013_VirtualString**)&pThread->StackPointer[7];
    T8013_VirtualString** temp24 = (T8013_VirtualString**)&pThread->StackPointer[8];
    T8013_VirtualString** temp26 = (T8013_VirtualString**)&pThread->StackPointer[9];
    T8013_VirtualString** temp27 = (T8013_VirtualString**)&pThread->StackPointer[10];
    T8013_VirtualString** temp33 = (T8013_VirtualString**)&pThread->StackPointer[11];
    T8013_VirtualString** temp34 = (T8013_VirtualString**)&pThread->StackPointer[12];

    memset(&pThread->StackPointer[1], 0, sizeof(void*) * 12);
    pThread->StackPointer += 12;

    // Value type locals
    Bool local0 = false;
    Char* temp2 = null;
    Char* temp5 = null;
    UInt32 temp6 = 0;
    Int32 temp7 = 0;
    Int32 temp8 = 0;
    Bool temp9 = false;
    UInt16 temp12 = 0;
    Int32 temp13 = 0;
    UInt16 temp14 = 0;
    Char* temp15 = null;
    Char* temp16 = null;
    Int32 temp17 = 0;
    Int32 temp18 = 0;
    UInt16 temp19 = 0;
    Int32 temp20 = 0;
    Int32 temp21 = 0;
    Bool temp22 = false;
    Char* temp25 = null;
    Char* temp28 = null;
    UInt32 temp29 = 0;
    Int32 temp30 = 0;
    Int32 temp31 = 0;
    Bool temp32 = false;
    UInt16 temp35 = 0;
    Int32 temp36 = 0;
    UInt16 temp37 = 0;
    Char* temp38 = null;
    Char* temp39 = null;
    Int32 temp40 = 0;
    Int32 temp41 = 0;
    UInt16 temp42 = 0;
    Int32 temp43 = 0;
    Int32 temp44 = 0;
    Bool temp45 = false;

IL_0000: // ldarg.0
    temp23 = pThis;

IL_0001: // call: Void .ctor()
    (*pThis)->T8003_VirtualObject::T8003_Ctor_VirtualObject(pThread, (T8003_VirtualObject**)pThis);

IL_0006: // nop
IL_0007: // nop
IL_0008: // ldarg.0
    temp24 = pThis;

IL_0009: // ldarg.1
    temp25 = pStr;

IL_000a: // stfld: Char* m_buffer
    (*pThis)->m_buffer = pStr;

IL_000f: // ldarg.0
    temp26 = pThis;

IL_0010: // ldc.i4.0
IL_0011: // stfld: UInt16 m_length
    (*pThis)->m_length = 0;

IL_0016: // ldarg.0
    temp27 = pThis;

IL_0017: // ldfld: Char* m_buffer
    temp28 = (*pThis)->m_buffer;

IL_001c: // ldc.i4.0
IL_001d: // conv.u
    temp29 = (Int32)(UInt32)(0);

IL_001e: // ceq
    temp30 = (temp28 == (Char*)temp29);

IL_0020: // ldc.i4.0
IL_0021: // ceq
    temp31 = (temp30 == 0);

IL_0023: // stloc.0
    local0 = temp31;

IL_0024: // ldloc.0
    temp32 = local0;

IL_0025: // brtrue.s: 2
    if (temp32 != 0)
        goto IL_0029;

IL_0027: // br.s: 36
    goto IL_004d;

IL_0029: // br.s: 15
    goto IL_003a;

IL_002b: // ldarg.0
    temp33 = pThis;

IL_002c: // dup
    (*temp34) = *(pThis);

IL_002d: // ldfld: UInt16 m_length
    temp35 = (*temp34)->m_length;

IL_0032: // ldc.i4.1
IL_0033: // add
    temp36 = ((Int32)temp35 + 1);

IL_0034: // conv.u2
    temp37 = (Int32)(UInt16)(temp36);

IL_0035: // stfld: UInt16 m_length
    (*pThis)->m_length = temp37;

IL_003a: // ldarg.1
    temp38 = pStr;

IL_003b: // dup
    temp39 = pStr;

IL_003c: // ldc.i4.2
IL_003d: // conv.i
    temp40 = (Int32)2;

IL_003e: // add
    temp41 = ((Int32)temp39 + temp40);

IL_003f: // starg.s: Char* pStr
    pStr = (Char*)(temp41);

IL_0041: // ldind.u2
    temp42 = ((UInt16)*(UInt16*)pStr);

IL_0042: // ldc.i4.0
IL_0043: // ceq
    temp43 = (temp42 == (UInt16)0);

IL_0045: // ldc.i4.0
IL_0046: // ceq
    temp44 = (temp43 == 0);

IL_0048: // stloc.0
    local0 = temp44;

IL_0049: // ldloc.0
    temp45 = local0;

IL_004a: // brtrue.s: -33
    if (temp45 != 0)
        goto IL_002b;

IL_004c: // nop
IL_004d: // ret
    goto Exit;

Exit:
    pThread->StackPointer -= 12;
}

IL_0041: // ldind.u2
temp42 = ((UInt16)*(UInt16*)pStr);

Since we talk that the stack contains values and not variables, how did pStr got in this statement? It should be temp38 instead.

7 hours ago,Dexter wrote

I'm baffled. How did your compiler generate this line?

1 2	IL_0041: // ldind.u2 temp42 = ((UInt16)*(UInt16*)pStr);

Since we talk that the stack contains values and not variables, how did pStr got in this statement? It should be temp38 instead.

Yea good question. I reverted the changes I made to generate the temp variables because that was obviously not the ultimate solution.

My current idea is to load the variable names onto the stack just like I did before, but then when the variable gets popped, to step back through all of the previous instructions (up until the point it got pushed) and see if anything modified the variable since it was pushed. If so, it will go back and swap it out for a temp variable. It will also swap it out for a temp variable if any instructions inbetween are targets for any branch instruction (in case other code modified the variable).

The above solution should cut down dramatically on the number of temp locals. But now until I implement that, I might not be able to get back to this problem.

I plan to copy the above code into a simple function and play around with the code to see if using temp variables in the right places solves the problem. I'm sure it it related to not using temp variables in the right places, just like you mention. Once I understand exactly what the code is supposed to look like, I will make sure my compiler does the right thing.

 to step back through all of the previous instructions (up until the point it got pushed)

Hmm, this is very easy to do when the push and pop are in the same basic block. If they're in different blocks then things can get complicate. One thing that you should note about these temporaries is that they have limited lifetime. Once a temporary is popped it can be reused to store another value.

IMO, you're trying to get things working and optimize the code at the same time. I think it's better to just get things working first.

10 minutes ago,Dexter wrote

*snip*

Hmm, this is very easy to do when the push and pop are in the same basic block. If they're in different blocks then things can get complicate.

Yes that is why I think I should take into account when branch instructions come inbetween the push and pop instructions. Better to err on the safe side and create temp variables. However I don't think this will add a lot of temp variables because what I find with IL code is that the instructions are grouped pretty tightly. It pushes a few variables, then immediately pops them to perform some operation. I have not seen many cases where it pushes something then goes off to do something else first.

One thing that you should note about these temporaries is that they have limited lifetime. Once a temporary is popped it can be reused to store another value.

Yes I plan to have a temp variable pool, and when a temp is popped, it is freed back to the pool. If a following instruction needs a temp of the same type, it will get that freed variable.

IMO, you're trying to get things working and optimize the code at the same time. I think it's better to just get things working first.

True, but I always feel that if you know an area of the code needs to execute fast, at least try to keep that in mind so that you don't have to come back later and completely rewrite everything because it was the wrong approach. Also note that one of the goals of this project is to see how much I can outperform the .Net Micro Framework, so some dabbling in various performance tests along the way to see how viable all of this is, is fine I think.

Yes I plan to have a temp variable pool, and when a temp is popped, it is free back to the pool. If a following instruction needs a temp of the same size, it will get that freed variable.

Why? The compiler will find it much easier to optimise your temporaries to use less registers and stack space (on release builds) than you will, but doing it manually will force to compiler to take worse optimisations.

Make sure you don't ever do performance testing on a debug build, or you'll get a really strange impression of what code is fast and what code is slow.

The C++ compiler will not be able to optimize my ref types away (and it shouldn't), because I store them on the virtual stack. Creating large numbers of temp ref types is not ideal because that will give more work to the GC, in addition to requiring a larger virtual stack (memory is quite limited on a microcontroller).

It should not be too difficult to implement a mechanism like I described to reduce the number of temp variables used. In addition, it makes debugging the C++ code easier if there aren't a huge amount of temp variables. Also remember that the C# method we are looking at is only about  6 lines of code, and it created 45 temp variables. This is going to get insane with complex code.

BTW, I had a typo in my previous post, which I fixed:

If a following instruction needs a temp of the same type [size -> type], it will get that freed variable.

Void VirtualString::Ctor_VirtualString(VirtualThread* pThread, VirtualString** pThis, Char* pStr)
{
    // Ref type locals
    VirtualString** temp0 = (VirtualString**)&pThread->StackPointer[1];

    memset(&pThread->StackPointer[1], 0, sizeof(void*) * 1);
    pThread->StackPointer += 1;

    // Value type locals
    Bool local0 = false;
    Char* temp1 = null;
    Char* temp2 = null;

IL_0000: // ldarg.0
IL_0001: // call Void .ctor()
    (*pThis)->VirtualObject::Ctor_VirtualObject(pThread, (VirtualObject**)pThis);

IL_0006: // nop
IL_0007: // nop
IL_0008: // ldarg.0
IL_0009: // ldarg.1
IL_000a: // stfld Char* m_staticBuffer
    (*pThis)->m_staticBuffer = pStr;

IL_000f: // ldarg.0
IL_0010: // ldc.i4.0
IL_0011: // stfld UInt16 m_length
    (*pThis)->m_length = 0;

IL_0016: // ldarg.0
IL_0017: // ldfld Char* m_staticBuffer
IL_001c: // ldc.i4.0
IL_001d: // conv.u
IL_001e: // ceq
IL_0020: // ldc.i4.0
IL_0021: // ceq
IL_0023: // stloc.0
    local0 = (((*pThis)->m_staticBuffer == (Int32)(UInt32)(0)) == 0);

IL_0024: // ldloc.0
IL_0025: // brtrue.s 2
    if (local0 != 0)
        goto IL_0029;

IL_0027: // br.s 36
    goto IL_004d;

IL_0029: // br.s 15
    goto IL_003a;

IL_002b: // ldarg.0
IL_002c: // dup
    (*temp0) = *(pThis);

IL_002d: // ldfld UInt16 m_length
IL_0032: // ldc.i4.1
IL_0033: // add
IL_0034: // conv.u2
IL_0035: // stfld UInt16 m_length
    (*pThis)->m_length = (Int32)(UInt16)(((Int32)(*temp0)->m_length + 1));

IL_003a: // ldarg.1
IL_003b: // dup
    temp1 = pStr;

IL_003c: // ldc.i4.2
IL_003d: // conv.i
IL_003e: // add
IL_003f: // starg.s Char* pStr
    temp2 = pStr;
    pStr = (Char*)(((Int32)temp1 + (Int32)2));

IL_0041: // ldind.u2
IL_0042: // ldc.i4.0
IL_0043: // ceq
IL_0045: // ldc.i4.0
IL_0046: // ceq
IL_0048: // stloc.0
    local0 = ((((UInt16)*(UInt16*)temp2) == 0) == 0);

IL_0049: // ldloc.0
IL_004a: // brtrue.s -33
    if (local0 != 0)
        goto IL_002b;

IL_004c: // nop
IL_004d: // ret
    goto Exit;

Exit:
    pThread->StackPointer -= 1;
}

Note at line IL_003f, a temp local is used to store the value of pStr before it is assigned a different value. At line IL_0048, that temp local is used instead of the now changed pStr. This fixes the bug with the off-by-one I was seeing previously.

I was wondering how to deal with debugging an application written for a microcontroller while using this .Net -> C++ compiler. Previously I was thinking that I would try to figure out what the .Net Micro Framework does to enable debugging, but I think that is a big task since I'll probably need to create a VS debugging plugin etc. Also it would be difficult (although not impossible) to reliable get the state of all variables etc while in the debugger.

So I thought of a completely different way to enable debugging using this .Net -> C++ compiler. Basically, the idea is to not even do the .Net -> C++ compilation at all during debugging. Instead, there will be a "core" (or kernel) component running on the microcontroller that contains all of the "internal" calls that you can make to, say, set or get the state of an IO pin. I already implemented "internal" calls in my compiler (which is not the same as PInvoke of course). So basically I can create a class in C# called InOutPin, which allows you to get or set the state of a specific pin. Inside this class, I have a method called, say, "GetPinState". I add a special attribute, like this:

[VirtualCallAttribute(IsInternalCall = true, FunctionName = "IOPins::GetState")]
bool GetPinState(int pinIndex)
{
    // Actual C# code that will execute while running in "debug" mode in VS.
    // This code will not be compiled to C++, instead it will call the specified C++ function directly
}

So what happens is that if you run the application inside VS, it will simply treat GetPinState like any other method and execute the code it contains. On the other hand, when that method gets compiled to C++ code, that attribute will tell the compiler that this is actually an internal call and will call the specified function mentioned in the FunctionName parameter. All of this is already implemented and works (I did a bunch of string functions, like creating, concatenating etc).

So my idea is to take advantage of that in the sense that I can add code into GetPinState (the C# version), that will make the same call into the microcontroller via a serial or USB port. So basically, when you start running the application in VS, the library you use will contain code to connect to the microcontroller which is running the kernel. That kernel will know when the debugger has connected and listen for any calls. Once the connection is made, if you then call GetPinState, the call will be remoted to the microcontroller which will then call the corresponding C++ function to get the pin's state.

I don't foresee any real reasons why such an approach won't work, but there are some pros and cons that I can think of:

Pros:

• You can use VS to debug, including using advanced features like edit and continue. As far as VS is concerned, it's just another full .Net application.
• Compilation and debugging should be fast, just a like any other .Net application (at least the parts that aren't calling into the microcontroller). You don't need to wait for your code to be compiled and first downloaded the microcontroller like you do now.

Cons:

• Remoted calls to the microcontroller might be slow (Netduino supposedly supports up to 3M baud). Not sure how slow this will be, but you probably won't be toggling pins at MHz speeds (the fastest speed you can toggle a pin in .Net MF on a Netduino is about 9KHz, compared to about 4-5MHz with native code on the same microcontroller).
• You won't be debugging the compiled C++ code, you will be debugging the pre-compiled C# code. Because of this, you will be relying on the assumption that the .Net -> C++ compiler won't introduce any bugs into your code. Which is unlikely. This may or may not be a problem.
• You won't know that the code you are writing isn't going to work on the microcontroller until you actually try to compile it with the special Release build. You might be using a class for which there isn't a version available, and you might be happily using WCF, XNA or WPF but only realize later on that it can't be compiled to C++.

Anyway to me this solution sounds much easier to implement, and unless I'm missing something big, I don't see how it won't be able to work.

If so, I need to get a new temp variable and assigne it the value of the variable, and also replace the previous variable with the temp variable everywhere it shows up on the stack.

Mmm, seems like a cool solution to me.

The C++ compiler will not be able to optimize my ref types away (and it shouldn't), because I store them on the virtual stack. Creating large numbers of temp ref types is not ideal because that will give more work to the GC, in addition to requiring a larger virtual stack (memory is quite limited on a microcontroller).

Then don't put them on a virtual stack. Why would adding more temporaries hurt the GC? If the C/C++ compiler detects that two temporary registers hold the same value then they alias them and you end up with fewer local variables. The C/C++ also agressively reuses registers and stack positions for variables that die before other ones become live on release build (this is the reason why debugging variables doesn't always work on release builds).

15 hours ago,evildictait​or wrote

*snip*

Then don't put them on a virtual stack. Why would adding more temporaries hurt the GC? If the C/C++ compiler detects that two temporary registers hold the same value then they alias them and you end up with fewer local variables. The C/C++ also agressively reuses registers and stack positions for variables that die before other ones become live on release build (this is the reason why debugging variables doesn't always work on release builds).

That won't work because the way the GC works is that it should be able to get to any object that is currently in use. If I create a new object, but don't put a reference to it on the virtual stack, and the GC does a Mark/Sweep operation, that object's Mark function will never be called (via enumerating the objects on the virtual stack), and hence it's Mark flag will never be set. Then during the Sweep phase, the object will be collected.

I don't believe the C++ compiler can or should optimize the ref types. Value types are fine, that can't cause any side-effects.

for (var idx = 0; idx < 10000000; idx++)
{
    var obj = new SomeClass();
}

The GC will dispose of the objects that are no longer rooted. It also makes a Finalize call into each object but right now it isn't yet hooked up to the class's destructor. That should be easy.

I was implementing boxing/unboxing but this exposed a flaw in my current compiler (boxing/unboxing works quite well). The problem is that code can branch somewhere into the "groupings" of instructions that ultimately becomes one of more lines of C++ code. While testing the following method:

static void Main(string[] args)
{
     var result = BoxTest(false);
}

private static int BoxTest(object obj)
{
    if (obj is bool)
        return (bool)obj ? 1 : 2;
            
    if (obj == null)
        return 3;
    else
        return 4;
}

I found that my compiled C++ version was not doing the right thing. Here is the generated C++ code: 

Void Usr_Program::Usr_Main(VirtualThread* pThread, InternalArray<VirtualString*>** args)
{
    // Ref type locals
    VirtualObject** temp0 = (VirtualObject**)&pThread->StackPointer[1];

    memset(&pThread->StackPointer[1], 0, sizeof(void*) * 1);
    pThread->StackPointer += 1;

    // Value type locals
    Int32 local0 = 0;
    Int32 valueRetVal0 = 0;

IL_0000: // nop
IL_0001: // ldc.i4.0
IL_0002: // box System.Boolean
    pThread->AllocObject(sizeof(VirtualBox<Bool>), (VirtualObject**)temp0);
    new (*temp0) VirtualBox<Bool>(0x0005, 0);

IL_0007: // call Int32 BoxTest(System.Object)
    valueRetVal0 = Usr_Program::Usr_BoxTest(pThread, temp0);

IL_000c: // stloc.0
    local0 = valueRetVal0;

IL_000d: // ret
    goto Exit;

Exit:
    pThread->StackPointer -= 1;
}

Int32 Usr_Program::Usr_BoxTest(VirtualThread* pThread, VirtualObject** obj)
{
    // Value type locals
    Int32 local0 = 0;
    Bool local1 = false;

IL_0000: // nop
IL_0001: // ldarg.0
IL_0002: // isinst System.Boolean
IL_0007: // ldnull
IL_0008: // cgt.un
IL_000a: // ldc.i4.0
IL_000b: // ceq
IL_000d: // stloc.1
    local1 = ((((obj && (*obj)->IsInstanceOf(0x0005)) ? obj : null) > null)) == (0);

IL_000e: // ldloc.1
IL_000f: // brtrue.s 15
    if (local1 != 0)
        goto IL_0020;

IL_0011: // ldarg.0
IL_0012: // unbox.any System.Boolean
IL_0017: // brtrue.s 3
    if ((((VirtualBox<Bool>*)(*obj))->Value) != 0)
        goto IL_001c;

IL_0019: // ldc.i4.2
IL_001a: // br.s 1
    goto IL_001d;

IL_001c: // ldc.i4.1
IL_001d: // stloc.0
    local0 = 1;

IL_001e: // br.s 19
    goto IL_0033;

IL_0020: // ldarg.0
IL_0021: // ldnull
IL_0022: // ceq
IL_0024: // ldc.i4.0
IL_0025: // ceq
IL_0027: // stloc.1
    local1 = ((obj) == (null)) == (0);

IL_0028: // ldloc.1
IL_0029: // brtrue.s 4
    if (local1 != 0)
        goto IL_002f;

IL_002b: // ldc.i4.3
IL_002c: // stloc.0
    local0 = 3;

IL_002d: // br.s 4
    goto IL_0033;

IL_002f: // ldc.i4.4
IL_0030: // stloc.0
    local0 = 4;

IL_0031: // br.s 0
    goto IL_0033;

IL_0033: // ldloc.0
IL_0034: // ret
    return local0;

}

What I found was that the unboxing call at line IL_0017 was not being called at all. In fact, the compiler didn't even bother to compile that part of the code at all (it's completely missing from the disassembly). Closer inspection shows that as far as the test on line IL_0017 goes, it will always end up going to line IL_001d, hence it doesn't even bother.

Obviously this is the wrong end result, because in the C# version of the method, the check for true/false has a real result on the return value of the method.

I'm trying to think of ways to change the compiler to produce correct code, but so far I haven't thought of a good solution yet. This is obviously a solvable problem because all IL -> native code compilers have solved this problem. There shouldn't be any reason why I can't create functional C++ code as opposed to functional native code (if anything it should be easier).

Anyway, one possible solution I've been thinking about is to take advantage of the fact that I've already done a pre-pass at this point and I know exactly which instructions are branch targets. I can use this information to break the instructions into smaller groups. The problem with this is that it is not always possible (or obvious) to break instructions into smaller groups. For instance, in this case I need to break the instructions between IL_001c and IL_001d (each one is a branch target). But how do I write IL_001c in C++ code?

I'm just curious, what method is used to compile IL to native code? I'm using an evaluation stack based approach that accumulates expressions until an instruction comes along that does some real work, like storing, comparing, branching etc, based on the evaluation stack contents. And only then it output the code to perform that operation. Is there a different approach? At least the evaluation stack seems logical to me since it follows the same evaluation stack based approach of the IL very closely.

Closer inspection shows that as far as the test on line IL_0017 goes, it will always end up going to line IL_001d, hence it doesn't even bother.

The test is not the problem, the problem is that the ldc 2 at IL_0019 is nowhere to be found in the generated C++ code. My guess is that the reason for this bug is the following:

I'm using an evaluation stack based approach that accumulates expressions until an instruction comes along that does some real work, like storing

The value (expression) that gets stored here was defined in two different places. Somehow you lost track of one of those definitions and ended up always writing 1. IL_001D can be reached with 2 different stack states but the generated code accounts for only for one.

I'm just curious, what method is used to compile IL to native code?

I don't know exactly what CLR does but what I'd do is convert the IL to some intermediate representation where there's no stack and where all instruction operands are explicit instead of implicit. Such explicit representation are far more easy to reason about.

You may say that you aren't writing a compiler but getting familiar with some compiler techniques would be a good idea. Reading the code generation chapter(s) from the "Dragon Book" (http://dragonbook.stanford.edu/) is advisable.

5 hours ago,Dexter wrote

*snip*

The test is not the problem, the problem is that the ldc 2 at IL_0019 is nowhere to be found in the generated C++ code. My guess is that the reason for this bug is the following:

*snip*

The value (expression) that gets stored here was defined in two different places. Somehow you lost track of one of those definitions and ended up always writing 1. IL_001D can be reached with 2 different stack states but the generated code accounts for only for one.

Yes that is what I was trying to say, and I understand the problem in this case. The IL sees the two instructions as two distinct locations to jump to, while in the currently compiled C++ code, it translates to just one location, hence the C++ compiler rightly sees the test as superfluous and removes it.

Interestingly, if you run the debug version of ILSpy, in addition the the "IL" and "C#" decompile options, it has an additional 53 (yes, I counted them) decompile options of various stages of transforms and optimizations. Here is what the method looks like when selecting the decompile option "ILAst (after InlineVariables)":

[sorry once again the C9 forum insists on putting everything on one line, so using plaintext]
--------------------------

arg_1D_0
var_0_1D : int32

    brtrue(IL_20, ceq(cgt.un(isinst([mscorlib]System.Boolean, ldloc(obj)), ldnull()), ldc.i4(0)))
    brtrue(IL_1C, unbox.any([mscorlib]System.Boolean, ldloc(obj)))

    arg_1D_0 = ldc.i4(2)
    br(IL_1D)

IL_1C:
    arg_1D_0 = ldc.i4(1)

IL_1D:
    stloc(var_0_1D, arg_1D_0)
        br(IL_33)

IL_20:
    brtrue(IL_2F, ceq(ceq(ldloc(obj), ldnull()), ldc.i4(0)))
        stloc(var_0_1D, ldc.i4(3))
        br(IL_33)

IL_2F:
    stloc(var_0_1D, ldc.i4(4))

IL_33:
    ret(ldloc(var_0_1D))

--------------------------

So it used a temp variable called IL_1D_0 to store the result of both expressions lldc.i4(2) and ldc.i4(1). So the variable was associated with the instruction at IL_1D. This is similar to what I was trying by assigning the expression to temp variables, all I have to do now is figure out how to know when to use the same temp variable as opposed to grabbing a new one each time.

private static int PerformanceTest3()
{
    var result = 0;

    for (var idx1 = 0; idx1 < 30000; idx1++)
    {
        var strResult = "";

        for (var idx2 = 0; idx2 < 1000; idx2++)
        {
            var intVal = 0;
            strResult = idx2.ToString();
            Int32.TryParse(strResult, out intVal);
            result += intVal;
        }
    }

    return result;
}

The release build of this the .Net method takes about 7 seconds to complete on my system. When I compile this using my .Net -> C++ compiler, the debug build takes about 45 seconds to complete, but the release build only takes 3 seconds to complete - more than twice as fast as the .Net version.

Initially I though some optimizations must have resulted in the code not doing the right thing, but I confirmed that even with the C++ version that runs for just 3 seconds, that the return value is correct (2100098112 in this case). There is no way the C++ compiler can optimize away the ToString and TryParse calls.

I am checking whether strResult is null in my Int32::TryParse call, and off the top of my head I can't see what else I should be checking. Also, I don't yet have a try/catch mechanism in place but I already have an idea how that will work and it should not add a lot of overhead. 

My guess is that the .Net version is probably getting bogged down while dealing with culture related conversions when converting the value from int to string and back.

Anyway, quite encouraging results.

I had to rewrite a lot of the code because it just wasn't flexible enough to handle these kinds of scenarios. The updated code allows one to go back and retroactively modify variables (change constants to temps, swap a set of stack values for a temp, etc).

This has complicated the code a lot but I guess that is typical of any type of compiler. Anyway, it seems to work fine now, and in the process it automatically fixed the issue mentioned above where the "jump to" location doesn't really exist and therefore the result could be incorrect.

Off to the next issue... I want to implement context switching, keeping in mind that ultimately this will run on a microcontroller that has just one thread. For now, I'll do this on x86 since I use it to test all of this stuff. Once I get it to work, I just need to have ifdef sections for the different supported processors.

For x86, I found this old web page. I'm not familiar with "AT&T assembly syntax", so I'm not 100% sure how to convert that to the Intel format that VS will understand. Here's my 1st attempt based on the linked page:

void VirtualMachine::SwitchContext(VirtualThread* pOld, VirtualThread* pNew)
{
    Int32 oldStack = (Int32)pOld->NativeStack; 
    Int32 newStack = (Int32)pNew->NativeStack; 
    
    __asm
    {
        mov eax, oldStack;
        mov ecx, newStack;

        // Save registers
        push ebp
        push ebx
        push esi
        push edi

        // Save old stack
        mov eax, esp

        // Load new stack
        mov esp, ecx 

        // Restore registers
        pop edi
        pop esi
        pop ebx
        pop ebp
        ret
    }
}

I'm a bit rusty on my x86 assembly, but what I don't understand is that the code is moving oldStack into eax, then a few lines later, esp is moved into eax. Is my translation of the original code even correct?

Also, there seems to be a bit of an chicken and egg problem here. Obviously this code should only be called to switch between two threads that have already been running previously. So what I'm missing right now is a way to set up these stacks for the initial run (in my simulated Thread.Start call, which is implemented except for the fact that it doesn't yet set up the native stack).

In addition, my assumption here is that my VirtualThread class will not only hold the "virtual stack", but also this native stack that will be used to do context switching for each thread. Up until now there was only one "thread" so it was just using whatever the stack was of the running thread.

Interestingly, the compiler gives me this warning:

warning C4731: 'VirtualMachine::SwitchContext' : frame pointer register 'ebp' modified by inline assembly code

I assume this is a good thing because that is exactly what the intention was? Or is this a legit warning that I'm actually doing something wrong?

Interestingly, the compiler gives me this warning:

I'll answer this first because the rest can't be done properly with this in the way. The compiler can generate prolog/epilog code for a function:

SwitchContext:
               push ebp         ; save the previous frame pointer
               mov  ebp, esp    ; setup the frame pointer
               sub  esp, 8      ; space for local variables
               ...              ; function code
               mov  esp, ebp    ; restore the stack pointer
               pop  ebp         ; restore the frame pointer
               ret

Now it should be obvious what's the warning about. The compiler uses ebp for its own purposes and warns you if you change it in inline asm. This compiler generated code can be a problem when you want to implement context switching because it affects the stack layout. You can implement SwitchContext even if such code is present but creating the stack for a new thread will be a problem because you don't know what stack layout SwitchContext expects.

A possible solution is to use __declspec(naked) (which prevents such code from being generated) and __fastcall (which causes the first 2 arguments of the function to be passed in registers ecx and edx).

I'm a bit rusty on my x86 assembly, but what I don't understand is that the code is moving oldStack into eax, then a few lines later, esp is moved into eax. Is my translation of the original code even correct?

Nope. In the original version there were some parathesis which you ignored. Something like mov %esp, (%eax) converts to mov [eax], esp.

Here's an example that uses __declspec(naked) and __fastcall):

__declspec(naked) void __fastcall SwitchContext(VirtualThread* pOld, VirtualThread* pNew)
{
    // N.B. the following code assumes that NativeStack is at offset 4 in VirtualThread. If that's not true
    // then the appropiate offset needs to be used when storing/loading the stack.
    __asm
    {        
        // Save registers
        push ebp
        push ebx
        push esi
        push edi
        // Save old stack
        mov [ecx+4], esp    // store to pOld->NativeStack
        // Load new stack
        mov esp, [edx+4]    // load from pNew->NativeStack
        // Restore registers
        pop edi
        pop esi
        pop ebx
        pop ebp
        ret
    }
}

So what I'm missing right now is a way to set up these stacks for the initial run

To create a new thread you have to allocate memory for its native stack and setup the stack exactly the same as in SwitchContext does. Once the thread stack is properly initialized you can simply call SwitchContext to start the new thread.

In addition, my assumption here is that my VirtualThread class will not only hold the "virtual stack", but also this native stack that will be used to do context switching for each thread.

Yes, if you create thread you'll need to also allocate the native stacks yourself.

void InitializeContext(VirtualThread* pNew, void *startFunction)
{    
    int stackSize = 4096;
    char *stack = reinterpret_cast<char *>(malloc(stackSize)); // allocate some space, malloc is just an example
    stack += stackSize; // go to the top of the stack because the stack grows down
    
    __asm
    {        
        mov ecx, esp // save current stack pointer
        mov esp, stack

        mov eax, startFunction
        push eax    // push the start function address as the return address (of SwitchContext)

        // Save registers
        xor eax, eax // let's always start a thread with zeroed registers
        push eax
        push eax
        push eax
        push eax

        mov stack, esp
        mov esp, ecx // restore the stack pointer
    }
   
    pNew->NativeStack = stack;
}

Most excellent, sir. I will try out your suggestions and let you know how it goes. Your help is greatly appreciated.

[EDIT] Just one observation... Since the native stack is growing down, I can probably share the same block of memory for both the virtual and native stack (so they'll be growing towards each other). Then I can easily do a check at strategic points to see how close the current virtual stack and native stack pointers are getting to each other. When they are within some predefined limit of each other, I can throw a stack overflow exception. This way I only need to deal with one area of "wasted space" per thread vs two, and a single check for stack overflow.

__declspec(naked) void __fastcall SwitchContext(void* pOld, void* pNew)
{
    __asm
    {        
        // Save registers
        push ebp
        push ebx
        push esi
        push edi
        // Save old stack
        mov [ecx], esp    // store to pOld->NativeStack
        // Load new stack
        mov esp, [edx]    // load from pNew->NativeStack
        // Restore registers
        pop edi
        pop esi
        pop ebx
        pop ebp
        ret
    }
}

Then I just call it like this (where both pOld and pNew are VirtualThread):

SwitchContext(&pOld->NativeStack, &pNew->NativeStack);

EDIT: I changed the title to reflect the current part of the project I'm working on.

void NativePlatform::StartMainThread(RuntimeThread* pThread, void* pData, void *pStartAddress)
{    
    void* stack = pThread->NativeStack;

    __asm
    {        
        mov eax, pThread 
        mov ebx, pData
        mov ecx, pStartAddress
        mov esi, ebp
        mov edi, esp

        mov ebp, stack
        mov esp, ebp

        push esi
        push edi
        push ebx
        push eax

        call ecx

        pop eax
        pop ebx
        pop edi
        pop esi

        mov esp, edi
        mov ebp, esi
    }
}

I need to pass in pThread and pData as two parameters into the start function. This seemed to work just fine until I started using std::wcout in my code, after which I get a heap corruption error when the application exits. If I call the start function directly without going through StartMainThread, then I don't get the corruption dialog when I exit. Obviously something is still wrong with the above asm code.

Any idea what I'm doing wrong?

void (*m_startAddressData)(RuntimeThread*, VirtualObject**);

You are thinking that maybe the calling convention is wrong?

Unlikely. It can be wrong only if you changed the default from cdecl to stdcall or fastcall.

Hmm, the code looks correct. The 4 pops after the calls aren't really needed but they don't hurt.

Heap corruption + stack allocated using malloc (I assume)... are you sure the intial native stack pointer is correct?

OK I finally was able to get back to the heap corruption issue. In order to really test context switching, I needed to implement threading, which required me to implement delegates (including MultiCastDelegates while I was at it), and also TimeSpan and DateTime. Quite a tangent.

Anyway, so I was testing the StartMainThread call as shown a few messages above. I tried many different things, including giving NativeStack its own block of memory, but that block of memory is always corrupted. I carefully looked at the end of the block of memory (where the new thread will push and pop values), and indeed it pushes them within the boundary of the allocated block of memory (going downwards as expected).

The corrupted memory is always the size of the allocated block of memory plus 0x40. It is as if the stack switching code is doing something with the end of the block of memory. I even tried allocating a huge block of memory, and then setting the NativeStack to the middle of this block of memory - it still ends up corrupting the end of the block of memory. It is as if it known where the boundaries of the allocated block of memory is and then mucks around at 0x40 bytes past the end of it.

I confirmed that other code that allocates different blocks of memory aren't corrupting those blocks by allocating large dummy blocks inbetween those allocation calls so that the used blocks are not next to each other - it always comes back to the NativeStack block of memory as the one being corrupted.

Any ideas what could be going on here?

1. you mentioned that it happens after using cout, what happens if you don't use it?
2. How many threads are you starting and from where? All from main thread? Or each thread starts another thread?

45 seconds ago,Dexter wrote

Hmm, I guess this will be "fun" to debug. Couple more questions:

1. you mentioned that it happens after using cout, what happens if you don't use it?
2. How many threads are you starting and from where? All from main thread? Or each thread starts another thread?

I found the problem (see edited message above). I really haven't started any additional threads at this point. I was just using the StartMainThread call to switch the stack of the main thread. It was the main thread (with its new stack) that was causing the corrupted memory due to the stack not being large enough.

Previously, without using wcout, there was no memory corruption which makes sense now that it is clear what the problem was.

I must say, working with C++ is really painful compared to C# (I used to do it a lot up to about 8 years ago when I started programming mainly in C#). Part of the problem is that the compiler tends to give you really obscure error messages when something goes wrong. Leave out a brace or semicolon somewhere, and it tells you some class in some unrelated file is undefined. Often you have to go the last error messages of dozens and dozens of others to figure out what the real problem is.

__declspec(naked) void __fastcall SwitchContext(void* pOldStack, void* pNewStack)
{
    __asm
    {        
        // Save registers
        push ebp
        push ebx
        push esi
        push edi
            
        // Save old stack
        mov [ecx], esp    // store to pOldStack
            
        // Load new stack
        mov esp, [edx]    // load from pNewStack
        
        // Restore registers
        pop edi
        pop esi
        pop ebx
        pop ebp
        ret
    }
}

My assumption here is that pOldStack and pNewStack should not the same as NativeStack for each thread, correct? NativeStack is really a constant value pointing to the top of the block of memory that was allocated for that thread's stack.

So instead, I need to have an additional void* in each thread whose address will be passed in to the SwitchContext call, correct? The question is, if this is the very first context switch, what should be stored at the pNewStack value? I assume this should be at some negative offset into that thread's NativeStack. Most likely I would need to calculate that value somewhere in the following call that prepares a thread for execution:

// Set up the thread so that it is ready to run
void NativePlatform::InitializeThreadContext(RuntimeThread* pThread, VirtualObject** pData)
{    
    void* stack = pThread->NativeStack;
    void* startAddress = (void*)pThread->GetStartAddress();

    __asm
    {        
        mov ecx, esp    // Save current stack pointer
        mov esp, stack

        mov eax, startAddress
        push eax        // Push the start function address as the return address

        // Save registers
        xor eax, eax    // Let's always start a thread with zeroed registers
        push eax
        push eax
        push eax
        push eax

        mov stack, esp
        mov esp, ecx    // Restore the stack pointer
    }
}

BTW, thanks for all of your help so far, it is greatly appreciated.

So instead, I need to have an additional void* in each thread whose address will be passed in to the SwitchContext call, correct?

Yes, this value contains the current stack pointer for the thread. When you switch away from a thread its current stack pointer gets saved and the stack pointer of the new thread is loaded.

The question is, if this is the very first context switch, what should be stored at the pNewStack value

That's the role of InitializeContext, it sets up the initial stack so it looks similar to the one expected by SwitchContext. That's why the start address gets pushed on the stack, when SwitchContext willreturn to what's stored on the stack, your start address. This might sound strange, the start function is notcalled but returned too, but it works fine as long as the stack is setup correctly. That's also the reason for those 4 push eax, they compensate the fact that SwitchContext pops edi, esi, ebp, ebx.

I assume this should be at some negative offset into that thread's NativeStack

The InitializeContext already computes the correct value but it looks like it doesn't store it correctly. Let's fix things a bit:

void NativePlatform::InitializeThreadContext(RuntimeThread* pThread, VirtualObject** pData) {
    void* stack = pThread->NativeStack;
    void* startAddress = (void*)pThread->GetStartAddress();

    __asm {
        // Save current stack
        mov ecx, esp

        // Switch to pThread's stack
        mov esp, stack

        // Push arguments for the start function
        // It looks like your start function expects pThread and pData so:
        mov eax, pData
        push eax      
        mov eax, pThread
        push eax

        // Push the start function address, SwitchContext will return to this address
        mov eax, startAddress
        push eax

        // Push 4 values to compensate for pop edi, esi, ebx, ebp in SwitchContext
        xor eax, eax
        push eax
        push eax
        push eax
        push eax

        // At this point the stack is similar to what SwitchContext expects, save the stack pointer
        mov stack, esp

        // Switch back to this thread's stack
        mov esp, ecx
    }
    // save the thread's current stack pointer
    pThread->CurrentStackPointer = stack;
}

I added some comments to make it more clear and also added arguments for your start function (my previous example didn't pass any arguments to the start function).

After InitializeContext runs you should be able to switch to the new thread by doing SwitchContext(&currentThread->CurrentStackPointer, &newThread->CurrentStackPointer).

Now there's another fun thing to solve. What happens when the thread exits?

I tried your code, and in that case it also successfully calls into startAddress, but EBP is null and so the stack is also messed up. I haven't yet been able to verify whether ESP is correct in this case.

That is kinda where I've been stuck, since I'm not 100% sure where to get the correct value to store for EBP. EBP is the last value it pops before executing the ret.

void NativePlatform::InitializeThreadContext(RuntimeThread* pThread, VirtualObject** pData)
{    
    void* stack = pThread->NativeStack;
    void* callObj = pThread->GetCallObject();
    void* startAddress = (void*)pThread->GetStartAddress();

    if (!callObj && !pData)
    {
        // Static call with no data
        __asm
        {        
            mov ecx, esp
            mov esi, startAddress
            mov eax, pThread
            mov esp, stack

            push eax
            push esi
            push esp

            // Save registers
            xor eax, eax             
            push eax
            push eax 
            push eax

            mov edi, esp
            mov esp, ecx

            mov stack, edi         
        }

        pThread->CurrentStackPointer = stack;
    }
    else if (!callObj && pData)
    {
        // ...
    }
}

About your observation related to how to handle the case when the thread exits. I was thinking about that too, and I think in that case it should trigger another context switch to another thread, and the context switch code just doesn't switch back to it after that. I could also push the necessary values onto the stack so that it ultimately "returns" into a function that causes a context switch, and hence switches to another thread.

but EBP is null and so the stack is also messed up

Well, it's null because that's how InitializeContext initializes it. But why do you care about the contents of EBP? The start function doesn't expect a value in this register.

Below is what I have so far

I see you're pushing ESP, why? Again, this will be poped as EBP but you don't need EBP to have any particular value.

OK I see what you are saying about EBP, although if I'm eventually going to put an additional return address on the stack to serve as a thread exit handler, I should probably have something meaningful in EBP.

But for now I'm not going to worry about EBP since the called function doesn't care about it, and instead try and figure out what else could be wrong here. As I said previously, once the called function is entered and I manually subtract 4 from EBP (after it was assigned from ESP in the prologue), the called function can see it's parameters properly. That might be a good clue as to where the problem could be.

Between push eax and push esi you need to add another push for the return address. This can be 0 if you chose never to return from the start function or it can be the address of an "exit function" that calls SwitchContext to get away from the thread.

Note I need 4 versions, depending on whether it is a static or instance call, and whether it passes the extra data or not

That sounds a bit excesive. Normally there is a single thread start function and this isn't the function you pass to the Thread constructor through a delegate. This start function is a special function that takes care of initializing the thread and then calls the delegate.

Also, the static/instance distinction is the job of the delegate, the thread start function should not be sensitive to this.

The four versions correspond to TreadStart and ParameterizedThreadStart where each of those can call a static or instance method. While the Thread constructor doesn't care about static vs instance, internally I need to handle those two cases. In this particular case, it determines how many parameters are pushed onto the stack. For instance functions, an additional "this" parameter will be pushed. In the generated C++ code, the four functions would look like this:

Void ThreadTestClass::ThreadTestStaticNoData(RuntimeThread* pThread) {}

Void ThreadTestClass::ThreadTestStaticData(RuntimeThread* pThread, VirtualObject** data) {}

Void ThreadTestClass::ThreadTestInstanceNoData(RuntimeThread* pThread, ThreadTestClass** pThis) {}

Void ThreadTestClass::ThreadTestInstanceData(RuntimeThread* pThread, ThreadTestClass** pThis, VirtualObject** data) {}

Those are the functions that the first context switch need to call into, and each have a different parameter signature.

EDIT: You have a good point about the single start function, I didn't think about that. So this single start function is the one that cares about the four different versions, and calls the correct one. In addition, it could do the cleanup after the start function returns. At that point it would set the Exit flag on pThread, and then call the scheduler which would switch away to another thread.

I just rewrote the code to use the single start function, but I have not yet fixed the off-by-4 bug. I have to leave for work right now so I'll have to wait to do that tonight maybe.

Next up is to implement Sleep and Join. Sleep should be easy in that all I need to do is set a future tick based on the specified millisecond value, then call into the scheduler to switch to a different thread. The scheduler will then enumerate through all threads and see which ones are ready to run, skipping those that have tick value set sometime in the future. BTW I'm using a counter on each thread that is incremented by the amount of its priority. Then the scheduler picks the thread with the highest counter value. When a thread is run, it's counter is set back to zero.

Join should also be easy in that it stores a pointer (or thread ID) to the thread it should "join". The scheduler will check whether the other thread is still in a running state and then keep skipping the joining thread until the other thread's Exited flag is set.

If no thread is ready to run, then my thinking was that I would need to call the asm halt instruction. However reading up on the halt instruction indicates that it is a privileged instruction that can only be executed by the kernel. I guess for x86 right now I can just call Sleep(0). If this eventually runs on a microcontroller I will just call whatever the equivalent is for the halt instruction.

<Tick, LinkedList<Thread>> 

I think the problem in this case will be that there can be various ticks in the table, but the actual tick might fall inbetween those values. Then the earlier ticks would be skipped altogether. Let's say the ticks are 1000, 1010, 1030. Then when the scheduler gets called, the actual tick at that time is 1005. That means that the thread that is supposed to wake up at tick 1000 will be skipped and it will sleep forever.

I don't expect too many threads to be created, so I don't think this area would be a bottleneck. Also, the scheduler needs to enumerate through all threads anyway so it is pretty cheap to add an additional check in that loop to see whether the thread should be woken up.

I am thinking of ways to optimize this area at a later point, so I appreciate your suggestions.

I have the main thread and 4 other threads that each outputs a console message in a loop and then Sleeps for a few hundred ms during each iteration. When each non-main thread's loop exits, it sets a flag and the main thread loops until all four flags are set.

Everything works fine when either the Debug or Release builds run in the debugger, but both the Debug and Release builds get access violations when run outside of the debugger. Obviously this makes it difficult to debug. I'm wondering whether the thread switching function is really saving enough registers. Maybe adding naked or fastcall to the function declaration/definition changes the assumptions about which registers need to be saved?

So apparently the exception that gets thrown when running the C++ app outside of the debugger is 0x40010006, which is DBG_PRINTEXCEPTION_C. When I comment out all OutputDebugString calls, the exception is no longer thrown and the application runs without problems.

According to the MSDN documentation, if the application is run outside of the debugger, OutputDebugString will output to the system debugger. If there is no system debugger, it does nothing. Except in my case it crashes the application.

I did increase the stack size a lot to ensure it isn't running out of stack space again.

Any ideas?

the exception that gets thrown when running the C++ app outside of the debugger is 0x40010006, which is DBG_PRINTEXCEPTION_C

Hmm, I have the feeling that Windows won't always like this stack switching business. NT's structured exception handling relies on the stack and if you switch the stack... I'm not quite sure what can happen.

I'm wondering whether the thread switching function is really saving enough registers

It should be enough unless you use SSE. A normal kernel the relise on interrupts to switch threads has no choice but to save all registers because it doesn't know which registers are used. But what you're doing here is more like cooperative multithreading and more importantly, it's all done in C. The C/C++ compiler expects that a function will preserve those 4 registers (ebp, ebx, edi, esi), all other general purpose registers can be modified by the function as needed. Also, the FPU stack is supposed to be empty when a function is called so it doesn't contain anything that needs saving.

Well so as it turns out the exception is thrown when you call OutputDebugString and there is no debugger available. This is not what the MSDN documentation says (it says the call does nothing), but this is actually used by hackers to test whether their malware is running under a debugger or not. If a debugger is present, no exception would be cought and they would exit in order to prevent reverse engineering.

Doing the following solved the problem:

if (IsDebuggerPresent())
    OutputDebugString(text);

Now the context switching works flawlessly.