Huh. My mom has a flip phone that messes everything up when the battery comes off so I figured that it would be the same for newer ones too.

Assuming you mean an older flip phone, those tended to have problems (they probably had no way to fetch the time compared to modern smartphones), but they’ve not been an issue for a while, in my experience.

I regularly swapped the batteries on my old Nokia/Samsung smartphones without running into clock problems at all, since they would synchronise the moment they connected to the network, and got internet access, if they didn’t keep the time in the first place.