> When asking “do LLMs reason” we are asking whether the action being performed is similar to

Very certainly not. We ask if the system achieves the goal.

"When we ask if the coprocessor performs floating point arithmetic, we ask if the system achieves the goal (of getting accurate results)". Not, "does the co-processor ask if we have a spare napkin".