There's also infinite screen width courtesy of Fitts' Law if you keep the mouse against the edge of the screen.
As for the rest, making close buttons only appear for the mouse would solve the fat finger problems from touch use.
But your mouse starts against the edge of the screen, so any unintended travel is naturally going to be away from the edge, so you can't really rely on Fitts to help.
I'm also not convinced that you having close buttons appear/disappear depending on what input device your using would work particularly well in practice either. Aside from being potentially confusing, it starts to introduce oddities with differing input devices (Should a stylus act more like a mouse or touch? What about Kinect? What about a mouse that supports Touch gestures?)