Limit voters to one polling location. Problem solved.

That's what we do in the US. You are assigned a polling location based on your home address. You can't vote anywhere else. If you try, they turn you away.

You can do a provisional ballot (for people who recently moved, and poll data isn't updated, etc) and they validate your ID/address/etc later.