If you go back in time and imagine yourself in 1950, would you have said that about computers in general? Yes their decisions are the result of pre programmed heuristics, but to the operator that can feel as opaque as AI. In fact in the early days of computing AI was used to refer to a broader swath of what we now call heuristics and machine learning.

Is a very sophisticated computer driven adaptive control system making decisions for the user? Arguably yes. If your car has a bunch of assist tech you are not driving. You are commanding a nonlinear control system.

AI today is quantitatively far beyond these systems but internally it is still just a giant decision table. It’s just a differentiable table that can be automatically programmed by back propagation at what would be unimaginable scale for manual implementation.

As soon as you put a machine between you and the target action that is more complex than what you can hold in your head and that has multiple layers or feedback loops, you are ceding fine grained direct control in favor of a simpler abstract control surface.

I will change my mind a little if I start seeing evidence of genuine volition, but as of now I suspect that’s a property of life not intelligence. One interesting result of AI when viewed as an experiment is: I think it proves that life and intelligence are separate phenomena and that consciousness and intelligence may actually be unrelated.

I have the hypothesis that consciousness (and therefore true volition) is a property of life. You have to be metabolic and tied in some intimate way to thermodynamics and the arrow of time. Solid state electronics doesn’t have this property. It’s unrelated to intelligence. A bacterium may be conscious.