Make C# DLL methods callable like C-style DLLs

UPDATE: Found a handy C# project template that does this with less fuss (it uses Cecil btw unlike the other solutions):

http://sites.google.com/site/robertgiesecke/

http://sites.google.com/site/robertgiesecke/Home/uploads/csharpprojecttemplateforunmanagedexports/UnmanagedExportLibrary.zip

EDIT: I was looking for a tool to automate this but the one I found didn't work so I posted hopefully better one below. This is useful with countless popular proprietary win32 apps that only support calling straight C DLL's. As it's not really supported in C# AFAIK it may not be wise to use on widely distributed dll's.

This article explains it's possible to hack C# dll so it can be called directly from native code. There's probably some reason why it's not supported directly without hacks but well if it works then that's fine.

http://wayback.archive.org/web/20081015000000*/http://www.csharphelp.com/archives3/archive500.html

But editing the IL after every compile isn't fun and if I create a dll that I don't edit well that makes the whole exercise pointless, I might as well used CLI wrapper then. I find that tends to complicate things and slow down the rapid first development iteration. I'm not sure yet if this is better or worse when it comes to later maintenance and debugging - it's probably best to keep the ilasm,ildasm,fusion.dll and this tool with the project that uses it and write a note.

EDIT2:

There's an interesting attribute in NET 3.5->

 [System.Runtime.InteropServices.AllowReversePInvokeCalls]

http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.allowreversepinvokecallsattribute.aspx

However it doesn't do much according to quote ref source quote I found in SO.

// To be used on methods that sink reverse P/Invoke calls.
// This attribute is a CoreCLR-only security measure, currently ignored by the desktop CLR.
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public sealed class AllowReversePInvokeCallsAttribute : Attribute
{
public AllowReversePInvokeCallsAttribute()
{
}
}

Whooh. It works. This is kinda exciting because there's ton of useful closed source apps that can only load native DLL's and I want to use C# to do stuff inside native apps. There's probably some caveats like the ones about loading CLR into Explorer but I think in the latest CLR there was something about making that work better?

Here's the code:

C# dll:

win32 console app (this code requires multi-byte character set setting set): 

and the modifications from the article linked in OP:

CSharpDll.il:

// ### CHANGE #### -> Change Image CoreFlag to COMIMAGE_FLAGS_32BITSREQUIRED to fix the potential WinXP pitfall
.corflags 0x00000002
// ### CHANGE #### -> Create a VTable entry which will contain the needed data to identify our function
.vtfixup [1] int32 fromunmanaged at VT_01

// ### CHANGE #### -> Create a data entry to hold the Virtual Address to our function
.data VT_01 = int32(0)

...

  .method public hidebysig static string
          SayHello(string name) cil managed
  {
    // ### CHANGE #### -> Specify wich VTable entry to use for this function
    .vtentry 1 : 1
    // ### CHANGE #### -> Export the method as unmanaged code with the alias "SayHello"
    .export [1] as SayHello

I whipped up this for the IL hacking in post-build command.  licensed to do whatever it happens to do.

//C# DLL post-build step: path/to/CNativeExportEnabler.exe "$(TargetPath)"
//put ildasm.exe,ilasm.exe,fusion.dll at same location (path/to/)

//add [System.Runtime.InteropServices.AllowReversePInvokeCalls] to a static method to mark it for export
//post-build command: ...\CNativeExportEnabler\bin\Debug\CNativeExportEnabler.exe "$(TargetPath)"
//to change ILASM debug or for x64: ...\CNativeExportEnabler\bin\Debug\CNativeExportEnabler.exe "$(TargetPath)" /debug=opt /x64
using System;
using System.Collections.Generic;
using System.Text;
using System.Linq;
using System.IO;
using System.Diagnostics;namespace CNativeExportEnabler
{
    class Program
    {
        static void Main(string[] args)
        {
            var prms = Environment.GetCommandLineArgs().ToList(); prms.RemoveAt(0);
            string debugprm = "";
            bool x64 = false;
            for (int i = 0; i < prms.Count; i++)
            {
                var ar=prms[i].ToLowerInvariant();
                if (ar.StartsWith("/debug")) { debugprm = prms[i]; prms.RemoveAt(i); }
                if (ar.StartsWith("/x64")) { x64 = true; prms.RemoveAt(i); }
            }
            var path = string.Join(" ", prms.ToArray());
            if (debugprm == "" && path.ToLowerInvariant().Contains(@"\debug\")) debugprm = "/debug";
            var dllf = new FileInfo(path);
            var tpth = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName) + Path.DirectorySeparatorChar;
            var psi = new ProcessStartInfo();
            psi.WorkingDirectory = tpth;
            psi.FileName = @"ildasm.exe";
            var iln = dllf.Name.Substring(0, dllf.Name.LastIndexOf('.')) + ".il";
            psi.Arguments = '"' + dllf.FullName + '"' + " /linenum /nobar /out:" + iln;
            psi.CreateNoWindow = true;
            psi.WindowStyle = ProcessWindowStyle.Hidden;
            var p = Process.Start(psi);
            while (!p.HasExited) System.Threading.Thread.Sleep(50);
            var ls = System.IO.File.ReadAllLines(tpth + iln);
            var nls = new List<string>(ls.Length);
            int exports = 0;
            var msb = new StringBuilder();
            bool inmethod = false;
            int corflpos = 0;
            for (int i = 0; i < ls.Length; i++)
            {
                var l1 = ls[i].Trim();
                if (corflpos == 0 & l1.StartsWith(".corflags"))
                {
                    corflpos = i;
                    nls.Add(@".corflags 0x0000000" + (x64 ? "8" : "2") + "    //  COMIMAGE_FLAGS_" + (x64 ? "64" : "32") + "BITSREQUIRED");
                }
                else nls.Add(l1);
                if (l1.StartsWith(".method"))
                {
                    inmethod = true;
                    msb = new StringBuilder();
                }
                if (inmethod)
                {
                    msb.Append(l1);
                    msb.Append(" ");
                    if (l1 == "{")
                    {
                        inmethod = false;
                        var mls = i+1;
                        for (int si = mls; si < ls.Length; si++)
                        {
                            i = si;
                            var l2 = ls[si].Trim();
  
                            if (l2.StartsWith(".custom") && l2.Contains("AllowReversePInvokeCallsAttribute"))
                            {
                                var metd = msb.ToString();
                                var metd2 = metd.Substring(0, metd.IndexOf('('));
                                var metdsp = metd2.LastIndexOf(' ') + 1;
                                var metdkeywords = metd2.Substring(0, metd2.LastIndexOf(' '));
                                if (!metdkeywords.Contains("specialname") && metdkeywords.Contains("static"))
                                {
                                    exports++;
                                    var methodname = metd2.Substring(metdsp, metd2.Length - metdsp);
                                    nls.Add(@".vtentry 1 : " + exports.ToString());
                                    nls.Add(@".export [" + exports.ToString() + @"] as " + methodname);
                                    break;
                                }
                            }
                            nls.Add(l2);
                            if (l2.StartsWith("IL_")) break;
                            if (l2.StartsWith("}")) break;
                        }
                    }
                }
            }
            nls.Insert(corflpos + 1, @".vtfixup [" + exports.ToString() + @"] int" + (x64 ? "64" : "32") + " fromunmanaged at VT_01");
            for (int i = 0; i < exports; i++)
            {
                var exs = (i + 1).ToString();
                var exc = exs.PadLeft(3 - exs.Length, '0');
                var s = @".data VT_" + exc + " = int" + (x64 ? "64" : "32") + "(0)";
                nls.Insert(corflpos + 2 + i, s);
            }
            System.IO.File.WriteAllLines(tpth + iln, nls.ToArray());
            psi = new ProcessStartInfo();
            psi.WorkingDirectory = tpth;
            psi.CreateNoWindow = true;
            psi.WindowStyle = ProcessWindowStyle.Hidden;
            psi.FileName = @"ilasm.exe";
            psi.Arguments = iln + @" /dll /quiet " + debugprm + " /nologo /resource:CSharpDll.res /out:" + '"' + dllf.FullName + '"'; // /debug 
            Process.Start(psi);
        }
    }
}

Yes I did something like this a while ago to allow C# DLLs to be used as VST plugins in audio applications. The VST spec is purely a C API so it doesn't know anything about managed code.

My solution runs as a post-build step and it modifies the decompiled IL so that the methods you mark with a special "DllExport" attribute are exposed as C entry points. This allows a single C# DLL, without any additional support DLLs (which is usually a C++/CLI DLL) to be called directly from unmanaged code. The .Net runtime would automatically be loaded the first time unmanaged code calls into one of these exported methods.

Now if I had to do it again I would use reflection, AssemblyBuilder etc (or maybe Cecil) to decompile/recompile the IL, but back then I just decompiled the IL and parsed through the text file to make the required changes (which is similar to what you are doing).

Anyway, I posted this in a thread back then, and also posted the code I used in that thread. See here. All you need to do to expose a managed method is this:

[DllExport("main", CallingConvention.Cdecl)]
public static IntPtr MyEntryPoint(IntPtr someParameter)
{
    // Managed code...
}

And I still don't see why this is not exposed all the way up to C#. The .Net framework supports this, but it is not exposed in C#. If this was supported in C#, it would make it much easier to write plugins for unmanaged applications for instance.

EDIT: BTW in my case I also allow you to specify a x86 and/or x64 option, so that you can end up with DLLs that can be called from both 32-bit and 64-bit unmanaged code. I needed this because audio applications typically these days come in both 32-bit and 64-bit versions.

EDIT EDIT: As an interesting side-note... You can't export a managed method in this way, and then use P/Invoke in another managed application to call into the exposed method as an umnanaged function. Somehow .Net knows the method you exposed is really a managed method, and prohibits it being used that way (maybe for security reasons). I ran into this problem when I created a managed audio host application that supports the unmanaged VST spec, and can load unmanaged VST plugins. As soon as it tried to load one of my managed VST plugins (that is supposed to look like an unmanaged DLL), it stopped working because of this issue. I found a work-around but forgot exactly what I did.

Cool, in the original posting before editing I asked about (you) that as I did remember this idea being mentioned here.

I haven't yet got into utilizing this much as I've had some weird results: If I build the DLL in Release config and the ILASM uses /debug to re-assemble it, the performance is 2x slower than a normal debug build. I haven't figured out how to reliably and easily say if the IL is a debug build or not, the methods I googled did not work and when the re-assembled IL is ILDASMed again, much information has vanished etc.

Given above I thought about Cecil but figured that I have better things to do and simply now look at the path to the DLL, if it has \Debug\, then I add /debug switch, and additionally allow to specify that as parameter (like /debug=opt).

BTW. Something I came by that should be kept in mind with this method - particularly chatty interfaces might be doable without the interop overhead in C++/CLI - or through some more hacking but this instruction seems to have lot of restrictions in the spec.

'calli' is a low-level IL opcode (the assembly language of .NET). It's used to call a function by address (not by name). It's emitted only by C++/CLI not by C#.

Transitions from managed to unmanaged code require a small amount of overhead to allow exceptions and garbage collection to correctly determine the execution context. On an x86 processor, under the best circumstances, these transitions take approximately 5 instructions per call/return from managed to unmanaged code. In addition, any method that includes calls with transitions incurs an 8 instruction overhead spread across the calling method's prolog and epilog.

This overhead can become a factor in performance of certain applications. For use in unverifiable code only, there is a mechanism to call from managed code to unmanaged code without the overhead of a transition. A "fast native call" is accomplished by the use of a calli instruction which indicates that the destination is managed even though the code address to which it refers is unmanaged.

Yea I remember running into similar issues, but don't remember the details. The code I posted handles the debug/release stuff properly. So you can specify in the command line whether you want a debug or release build. This way, you can even debug the code while an unmanaged app calls into your managed code. Just specify the native app in the "Start external program" in the debug settings. The only thing that doesn't work of course is Edit and Continue, but all other debugging features work as expected.

As far as I know with that code, there is no performance issues since I've used the resulting code to create a plugin for an audio application that is supposed to be "realtime" and for which performance is quite important.

BTW I think I might have fixed various bugs since I posted that code. Here is the latest code I have. There is also a compiled DllExport.exe file in case you just want to use it without first compiling the project.

Thanks, if my solution breaks down I'll switch to yours. I found some interesting notes about "calli" instruction that would allow C# to call native code without any overhead. That might be suitable for something like ASIO playback with tiny buffer?

Google for:

14.5.6.3       Fast Calls to Unmanaged Code

But now you are talking about calling unmanaged code from managed code as opposed to the other way around.

Yes I think the calli instruction is used to make "internal" calls, which is native calls as you mention. In this case though you have to make sure that the parameters are pushed onto the stack in the convention that the native function expects. Also how are you going to get the native pointer in the 1st place? I think now you are talking about generating your own IL which is quite a different story.

From my experience, using P/Invoke with parameters that don't require special conversion (like strings etc), is very fast and adds insignificant overhead. In my music host application, I implemented ASIO support and it works very fast. IIRC, I got stable ASIO IO with ~2ms buffers (2 input channes and 6 output channels) with about 25 signal channels. It is also a multithreaded host so different parts of the signal path get processed by different threads. As a reference, with an empty project, each complete buffer process takes an average of 6us. This includes reading the 2 input buffers and writing the 6 output buffers in the ASIO driver.

Surprisingly the GC doesn't seem to cause problems. I read somewhere that the CLR 4.0 GC is concurrent which explains the ability to get really low latencies without dropouts.

That was a bad example in hindsight, the audio is too low frequency app for such optimizations. There was some new stuff in WinRT that sounded like it could improve certain interop scenarios.

That call comes with a huge disclaimer/limitations so I think I'll first look at what new tricks there are in NET 4.5 for interop performance. I think there was some. I haven't yet got far enough to profile my current project but on first sight I could see the interop being a bottleneck as the native app is going to be spamming the C# dll with calls as fast as CPU allows. I don't know for sure until I've written all the C# logic to see where the bottlenecks are.

I think if you do some tests you will find that P/Invoke adds very little overhead, as long as your parameters are all simple blittable types. I think you will be hard-pressed to come up with a custom solution that improves performance enough to make it worth the extra effort.

1 hour ago,Ion Todirel wrote

or you can just use managed C++, or simply COM which are far more flexible, rather than to rely on hacks

Neither managed C++ nor COM solve this particular problem.

For one, if you are dealing with an established C API that does not use COM (like VST for instance), you can't use COM at all. How would you do that?

In addition, sometimes you need a solution that uses just a single DLL without the additional C++/CLI DLL. VST plugins usually consist of a single DLL. That is just the convention.

At least those were the constraints in my case. I'm not 100% sure what androidi's requirements are, but it sounds like something similar. Yes this isn't such a common use case but it doesn't mean it is therefore invalid.

I think it's very common case when just something simple is enough and in software written in C and scripting languages which allow importing of C-style DLL's. And there's a lot of those out there.

I found this attribute in NET 3.5

[System.Runtime.InteropServices.AllowReversePInvokeCalls]

I'm not sure what MS does with it but since this IL mod solution is about hacking, I took the liberty of using that attribute to mark what methods are "allowed for reverse pinvoke from native code". When the attribute is found on a static method, I replace it with the export stuff.

@androidi:

That is a really interesting find. Unfortunately the documentation doesn't say anything useful about it, and there is no sample code for it. Web searches also don't bring up too much. The best use of it I could find is from this page. Specifically it shows how you can specify the calling convention, which is really important in this case.

When I have some more time, I will play around with it.

EDIT: It seems like the AllowReversePInvokeCalls attribute might only be useful to change security attributes of a method. Maybe this is not used to specifically export the method as a C function, but to allow the scenario I mentioned above where I could not get a round-trip to work.

@BitFlipper:I don't think that's a legit example, it has DllImport there. AFAIK the AllowReversePInvokeCalls doesn't do anything and I edited my OP in this thread with comments about it I found on SO. http://stackoverflow.com/questions/7018677/what-does-the-allowreversepinvokecallsattribute-do

BTW. I have a question in Tech off about seeking a file playing in WMP from another process, if anyone here has some workable pointers to get that going I'd appreciate those even if they don't fit the "ideal" that I put forth.

Yes I just tested this and it doesn't export the method as a C function. But as I said, I think this could help me at least where I needed to be able to do a round-trip (P/Invokde -> Reverse P/Invoke).

Just updating the OP:

I found this project template which uses Cecil and creates project with ready made export attribute and msbuild rule to fix up the exe:

http://sites.google.com/site/robertgiesecke/

http://sites.google.com/site/robertgiesecke/Home/uploads/csharpprojecttemplateforunmanagedexports/UnmanagedExportLibrary.zip