My main piece of advice is that you should get yourself a proper performance testing suite and perf-test your application to find out where it's slowing down. Premature optimisation is the root of all evil and leads to security bugs and nightmarish code, so don't do it if you don't have to, but in most apps 95% of the CPU is taken up on 5% of the code - some small tweaks in the hotpaths of your application could massively improve its performance.

If you're really stuck for ideas of how to speed it up, I'd consider migrating to Direct3D or XNA so that things like caching and fast access to the hardware are abstracted further away from you - Direct2D is really great for some things like ClearType font rendering and smooth scaling of splines, images and lines - if you don't need these things then move post-haste to Direct3D where these things are not in the way slowing down your application.

You might also want to consider checking if you're creating new objects and memory allocations every frame which could be cached across different frames - often these small improvements can make your application much faster.