Yes, and it immediately called to mind for me the phrase “the map is not the territory.”

Put another way: no matter how detailed or “perfect” you make a map, it will never be the territory, ie the thing that is mapped.

Computers and AI are like a map in this regard —- just ones and zeros that we have assigned meaning to arbitrarily. No matter how “good” AI gets, it’s still just a map of the thing not the thing itself.

So AI saying “I feel sad” is never more than a representation of sadness that should not be confused with the subjective experience of sadness itself.

If you make a big enough map you can fly it over and drop it on the territory I guess. Then does it become the territory?