There is a "simpler" way, which is still pretty stupid. Hopefully MS will fix this soonish, I actually kind of like the new mail client (and the Modern UI part of Win8 in general) but these oversights are pretty shocking !
Anyways, the fix:
disable your network interface(s) (Win key+W, type "view network connections", right-click and Disable your network connection)
open the Mail app and go to the outbox
there is now a delete icon in the top right for each stuck mail item !