Way easier to explain it this way:

Switching turns a correct guess into an incorrect one, and an incorrect one into a correct one. Your initial guess was more likely incorrect.