> Here’s a thought experiment: suppose that a mathematician solved a major problem by having a long exchange with an LLM in which the mathematician played a useful guiding role but the LLM did all the technical work and had the main ideas. Would we regard that as a major achievement of the mathematician? I don’t think we would.

This is a cultural choice. It makes sense that in the mathematics culture we currently have, this is alien. But already, other fields, and many individuals, would disagree and say that the human did have a major achievement here. As long as human-AI collaborations are producing the best results, there is meaningful contribution by the humans, and people that are deeper experts and skilled LLM whisperers should be able to make outsized contributions. The real shoe drops when pure AI beats humans and human-AI collaboration.

I replied to a comment about AI in sports and I build on that.

We praise car drivers despite most of the performance in their sport comes from the car. The driver makes the difference when two cars are close in performance. Brilliances or mistakes. Horse riders too.

In the case of math, the human can lead the LLM on the right track, point it to a problem or to another one. So it deserves some praise.

Then the team that built the car, cared about the horse, built the AI might deserve even more praise but we tend to care more about the single most visible human.

It may not be a major achievement by the mathematician (although it's debatable) but it would still be a major result.