That's really not how probabilistic systems work though. They're great at getting 80% of the way to the goal but the last 20% is always excruciatingly difficult. I really don't intend that a knock because LLMs are useful, but they also aren't magic boxes where anything is possible.

This is the problem that a lot of user-content based sites, and open software projects, are experiencing. The tools make prodiving 80% correct submissions so easy that many people are doing that and thinking that it is fine not to bother verifying for correctness (or not even knowing how to verify correctness). This means those projects are being inundated by submissions that are 20% tosh or worse.