Oh!

Usually it only continues to play for a few seconds after the window is destroyed for me.