@DaveWill2: The problem with a non-Form panel is that I don't believe it can go out of the bounds of the parent window (or at least it will be clipped). Tooltips can go off the edge of the parent form.
@ lensman: I tried that but get an exception during the Show call that a Form cannot be displayed if it is disabled.
In the end I did find a working combination of SetWindowLong(WS_EX_NOACTIVATE), ShowWindow(SW_SHOWNOACTIVATE), overriding Form.ShowWithoutActivation and returning true, and override CreateParams and returning WS_EX_NOACTIVATE.
Strangely if I leave any of the above out it stops working properly. Like how many times do you need to tell it not to activate? It's working the way I want it to now so I'm just going to leave it as is.