W3bbo has a point, porting computation intensive native code into managed code MIGHT introduce 'increased memory usage', 'delays in streaming' or 'stuttering audio as the result of GC' etc, that is true, but 'strange echoing' ? that is very unlikely, 'different behaviour' and 'performance issues' are different here.
Does your audio codec have an official test suite/ test vector ? does your native version enable some additional 'echo-cancelling' mechanism ?
I have used NSpeex and iLBC.NET before, without major problems, except slight performance degrading though.
And, native code on Windows Phones might be an Apollo thing I guess. and I have a feeling that you wont get it with a CE kernel,