If the use of an SDL enumerator rather than an integer indexer is the thing tanking your performance, then something has gone so far wrong in your app it's unreal.
Wow that is quite a generalization for someone that has no clue what my code looks like. I'm talking about tight loops that does DSP processing (realtime FFT, filtering, FFT analysis, etc) as well as code that does asynchronous multithreaded low-level IO. In such cases the overhead of using enumerators can easily add a 30% performance drop in such tight loops. I've done the benchmarks and that is what they show. What does your benchmarks show for such cases?
If you think "something has gone wrong" in my code then with all due respect you should stick to writing simple code where no-one gives a crap about actual performance.