I hate that framing because it glosses over what makes AI distinct from other technology. Pretty much anything can be used as a tool, that’s not an interesting aspect of AI. However standard tools do not take decisions on behalf of the user, they do not apply judgement. They come with assumptions and constraints that make some decisions more likely to be made. But the whole thing about LLMs is that they apply judgement and take decisions. When you use an LLM you delegate both aspects to it with very little overview or control.

I don’t think it’s “just a tool”, it’s something that directly compete with the role of humans in a system, by design

If you go back in time and imagine yourself in 1950, would you have said that about computers in general? Yes their decisions are the result of pre programmed heuristics, but to the operator that can feel as opaque as AI. In fact in the early days of computing AI was used to refer to a broader swath of what we now call heuristics and machine learning.

Is a very sophisticated computer driven adaptive control system making decisions for the user? Arguably yes. If your car has a bunch of assist tech you are not driving. You are commanding a nonlinear control system.

AI today is quantitatively far beyond these systems but internally it is still just a giant decision table. It’s just a differentiable table that can be automatically programmed by back propagation at what would be unimaginable scale for manual implementation.

As soon as you put a machine between you and the target action that is more complex than what you can hold in your head and that has multiple layers or feedback loops, you are ceding fine grained direct control in favor of a simpler abstract control surface.

I will change my mind a little if I start seeing evidence of genuine volition, but as of now I suspect that’s a property of life not intelligence. One interesting result of AI when viewed as an experiment is: I think it proves that life and intelligence are separate phenomena and that consciousness and intelligence may actually be unrelated.

I have the hypothesis that consciousness (and therefore true volition) is a property of life. You have to be metabolic and tied in some intimate way to thermodynamics and the arrow of time. Solid state electronics doesn’t have this property. It’s unrelated to intelligence. A bacterium may be conscious.