You just need a cheap ceiling fan balancing kit to fix that. Pretty much every fan comes with one, but almost everyone skips that step in the instructions during installation.