Information is an upper bound on knowledge.

> 10 billion year training period, something many orders of magnitude more expensive than an LLM.

I'm saying both definitions are valid definitions, they both point to important and different things: skill now, vs. how hard it is to get new skills. Some would describe it as "crystallised intelligence vs fluid intelligence".

I think it's important that any arguments are over the thing in dispute, not the label for that thing. Don't mistake the map for the territory.

Anyone who says "AI is stupid" by the first definition, what it can do, I think is making an error: they are already wildly super-human in at least some areas, if not generally.

Anyone who says "AI is stupid" by the second definition, how many examples they need, I agree with: there is a lot they are not currently able to learn even though it is easy for us, because the data they would need to do the learning on does not exist at the scale they need.

Also note: examples, not years. An alien intelligence whose synapses trigger 10 times faster or slower than mine (or ten million times faster or slower than mine), but who gets as much as I do out of each book or conversation, is my equal by the second definition.

I wouldn't say that information is an upper bound on knowledge because we don't measure knowledge in bits. The number of possible sequences of N bits is 2^N and knowledge involves selecting the sequences that are useful in some way. I don't know how to quantify it, but in principle it could be much larger than N.

I don't think I agree with your characterization of the second definition. Time scales matter. It's not much use to be able to solve human-scale problems if it takes millennia. And it only takes months to train an LLM to the level that it can solve cutting-edge math problems.