I need speakerphone when I'm home alone and attempting to be on a phone call while doing other things. Some of those calls are even about instructing me to look for something, so it necessitates me to be moving about. Speakerphone is an incredibly useful utility.

Don't take functionality away because of a few bad actors. That'd be like getting rid of drones because a few people are assholes.

Put rules in place to correct the bad behavior. Kicking them off planes seems fair.