At any rate, if the AI's side in this conversation were a human, that would be an extremely intelligent human indeed.

But there's no way the thinking times would have been that short, of course.

Yes, trying to communicate (or watching others try to communicate) about these topics is incredibly frustrating because it's pretty much impossible to make any progress without interrogating people's different definitions, but nobody wants to do that because it would mean being pedantic, splitting hairs, etc.

It's not like this is a new problem. Turing had a definition most of a century ago, he wasn't the first and certainly wasn't the last. I don't think we need new terms necessarily, and I doubt we're all going to agree on a definition tomorrow.

> consciousness may be just another emotion: happiness, sadness, egoness.

It's clearly much more than that.

I'd say an entity capable of instructing one of the leading mathematicians of his era is pretty clearly intelligent by any reasonable measure - however it might be arriving at its output.

If natural language was structured in a logically computable way, we'd have had interesting chatbots by the late 80s, basically as soon as a dictionary fit in local RAM, and for the same reason we got compilers.

Da hole raisin y nat-lang be v. hard is dat i kan rite lik dis an it be cool 4 native engrish speekrs 2 unerstand. LLMs are of course fine with this sentence in exactly the way that Zork's engine couldn't be.

That's just defining intelligence or consciousness as whatever we specifically can't put in a machine. It's not a very useful definition. (And it's one that completely breaks down to nothing if we do manage to fully put these things into machines or somehow prove it's possible.)

> There is about 150 years of cognitive science experimentation in animals

Yeah that would be relevant if AI were an animal...

As I said, it's clearly intelligent, but a quite different intelligence to that shared by animals.

> If AI researchers cared about scientific thinking, they would be intensely focused on the brains of bees.

Basically every academic AI researcher in history was doing what you described. The AI industrialists stopped caring 6 years ago once they realized LLMs seem to have been the only thing in 80 years that actually seems to work at any useful level.

There are plenty of pioneering scientists who are either returning to actual AI research (Yann Lecun, Ilya, etc), and plenty who never left (Richard Sutton) who are doing exactly what you are talking about.