The point of my argument is that the vast majority of tasks we carry out do not require good reasoning, because if they did most humans would be incapable of handling them. The point is also that a whole lot of people claim LLMs can't reason, based on setting the bar at a point where a large portion of humanity wouldn't clear it. If you actually benchmarked against average humans, a whole lot of the arguments against reasoning in LLMs would instantly look extremely unreasonable, and borderline offensive.

> Tin foil hat says that it’s because it probably wouldn’t look great and most LLM research is currently funded by ML companies.

They're currently regularly being benchmarked against expectations most humans can't meet. It'd make the models look a whole lot better.