The better way of putting it is that collusion is ultimately limited by the ease of entry into and exit from the market. The existing players don't need to "suddenly" drop the prices when a new entrant appears, because they've been doing that already.

It's only when supply bumps into short-term capacity constraints and price must rise enough to fully ration demand that this assumption fails. But then even if a new entrant appears that's no reason to drop prices unsustainably, because why would you? They're not taking any share of the market away from you, they're serving new demand.