Really? that is odd. I am posting right now using ie8 on xp and everything looks fine. Also tested ie7 on Vista and it was fine too. All of the icons are showing up no problem.
I wonder if the font that is used generate the icons is being blocked somewhere.
Could you take a look at the tinymce site and see if the icons are showing up for you there? That page has almost all of the icons that we are using. Just trying to determine where this is breaking. thanks