Discussions

the exception that gets thrown when running the C++ app outside of the debugger is 0x40010006, which is DBG_PRINTEXCEPTION_C

Hmm, I have the feeling that Windows won't always like this stack switching business. NT's structured exception handling relies on the stack and if you switch the stack... I'm not quite sure what can happen.

I'm wondering whether the thread switching function is really saving enough registers

It should be enough unless you use SSE. A normal kernel the relise on interrupts to switch threads has no choice but to save all registers because it doesn't know which registers are used. But what you're doing here is more like cooperative multithreading and more importantly, it's all done in C. The C/C++ compiler expects that a function will preserve those 4 registers (ebp, ebx, edi, esi), all other general purpose registers can be modified by the function as needed. Also, the FPU stack is supposed to be empty when a function is called so it doesn't contain anything that needs saving.

OK, but did you try the fix for the start function? Does it work?

Note I need 4 versions, depending on whether it is a static or instance call, and whether it passes the extra data or not

That sounds a bit excesive. Normally there is a single thread start function and this isn't the function you pass to the Thread constructor through a delegate. This start function is a special function that takes care of initializing the thread and then calls the delegate.

Also, the static/instance distinction is the job of the delegate, the thread start function should not be sensitive to this.

Stupid me, I'm talking about the way threads exit and I forget that the return address of the start function needs to be pushed onto the stack to. Even if the start function never returns because it does a SwitchContext you still need to push it because otherwise the start function can't find its arguments.

Between push eax and push esi you need to add another push for the return address. This can be 0 if you chose never to return from the start function or it can be the address of an "exit function" that calls SwitchContext to get away from the thread.

but EBP is null and so the stack is also messed up

Well, it's null because that's how InitializeContext initializes it. But why do you care about the contents of EBP? The start function doesn't expect a value in this register.

Below is what I have so far

I see you're pushing ESP, why? Again, this will be poped as EBP but you don't need EBP to have any particular value.

So instead, I need to have an additional void* in each thread whose address will be passed in to the SwitchContext call, correct?

Yes, this value contains the current stack pointer for the thread. When you switch away from a thread its current stack pointer gets saved and the stack pointer of the new thread is loaded.

The question is, if this is the very first context switch, what should be stored at the pNewStack value

That's the role of InitializeContext, it sets up the initial stack so it looks similar to the one expected by SwitchContext. That's why the start address gets pushed on the stack, when SwitchContext willreturn to what's stored on the stack, your start address. This might sound strange, the start function is notcalled but returned too, but it works fine as long as the stack is setup correctly. That's also the reason for those 4 push eax, they compensate the fact that SwitchContext pops edi, esi, ebp, ebx.

I assume this should be at some negative offset into that thread's NativeStack

The InitializeContext already computes the correct value but it looks like it doesn't store it correctly. Let's fix things a bit:

void NativePlatform::InitializeThreadContext(RuntimeThread* pThread, VirtualObject** pData) {
    void* stack = pThread->NativeStack;
    void* startAddress = (void*)pThread->GetStartAddress();

    __asm {
        // Save current stack
        mov ecx, esp

        // Switch to pThread's stack
        mov esp, stack

        // Push arguments for the start function
        // It looks like your start function expects pThread and pData so:
        mov eax, pData
        push eax      
        mov eax, pThread
        push eax

        // Push the start function address, SwitchContext will return to this address
        mov eax, startAddress
        push eax

        // Push 4 values to compensate for pop edi, esi, ebx, ebp in SwitchContext
        xor eax, eax
        push eax
        push eax
        push eax
        push eax

        // At this point the stack is similar to what SwitchContext expects, save the stack pointer
        mov stack, esp

        // Switch back to this thread's stack
        mov esp, ecx
    }
    // save the thread's current stack pointer
    pThread->CurrentStackPointer = stack;
}

I added some comments to make it more clear and also added arguments for your start function (my previous example didn't pass any arguments to the start function).

After InitializeContext runs you should be able to switch to the new thread by doing SwitchContext(&currentThread->CurrentStackPointer, &newThread->CurrentStackPointer).

Now there's another fun thing to solve. What happens when the thread exits?

He he, I was wondering about the stack size but I didn't expect cout to need that much. I'm glad it's working now.

Hmm, I guess this will be "fun" to debug. Couple more questions:

1. you mentioned that it happens after using cout, what happens if you don't use it?
2. How many threads are you starting and from where? All from main thread? Or each thread starts another thread?

"You are thinking that maybe the calling convention is wrong?"

Unlikely. It can be wrong only if you changed the default from cdecl to stdcall or fastcall.

Hmm, the code looks correct. The 4 pops after the calls aren't really needed but they don't hurt.

Heap corruption + stack allocated using malloc (I assume)... are you sure the intial native stack pointer is correct?

How is startAddress function declared?