True - but a phone call scales much easier than driving to someone's house with a gun.

I've been told that scammers aren't interested in making scams too good, the idea being that you want to select for people who are bad at recognizing a mediocre scam, because they'll be more likely to play along for the entire scam.

Not just scale either. On the phone you're dealing with people having less fear from local repercussions, from reprisal, less care for the community, etc.