I believe they are free to challenge the decision in an administrative court in the US (under Attorney General, not an Article III court) but they can be arrested. In any case, most countries would reserve the rights to kick you out. That's the deal you make with the country when you choose to get their visa and go there.

I don't think the controversy is about deporting people who are no longer wanted in the country. It's about the use of unnecessary force. The principle of minimum necessary force is pretty integral to Western societies. It includes the idea that the authorities are not allowed to arrest anyone until more reasonable options have been exhausted.