won't they able able to track you down if you start using the SIM though?

They can track the locations where you use that card, and the locations where you use the phone the card was in, before and after the card was in it.

Take this information as you will.

ETA: "use" means "have a battery inserted"