There is a "simpler" way, which is still pretty stupid. Hopefully MS will fix this soonish, I actually kind of like the new mail client (and the Modern UI part of Win8 in general) but these oversights are pretty shocking !
Anyways, the fix:
- disable your network interface(s) (Win key+W, type "view network connections", right-click and Disable your network connection)
- open the Mail app and go to the outbox
- there is now a delete icon in the top right for each stuck mail item !
- don't forget to enable your connection(s) again