It's a feature. The monitor colour profile tells Windows how images should be adjusted to look natural and Photo Gallery is obeying that. Computer monitors have a tendency to be far too blue, but because you're used to that the proper colours will look
yellow by comparison. Hold a blank piece of paper next to a white area on your screen, and you'll notice the paper looks yellow by comparison, even though they're both supposed to be white.
That said, I don't much like it either; if the colour is to be corrected, it should be done globally. Because it's not, and only done to the images, they look out of place like you noted.