You're confusing collusion with being informed. The concept of market rationality is based on the premise that all participants in said market more or less have access to the same information. Fools can choose to not be informed before making a trade, but passing along sensitive information that contradicts market rational behavior causes people to lose trust in the market.

Perfect example from today. Allbirds just announced that they're going all in on AI infra, skyrocketing the stock. Had I bought a million dollars worth of Allbirds yesterday, everyone would think I'm an idiot. But now, they would think I have insider information and would no longer want to participate because it would make no sense to buy Allbirds yesterday unless I knew the announcement was coming.