> After all, medicine is all about knowledge, experience and intelligence

LLMs are really really good at knowledge.

But they are really really bad at intelligence [0]

They have no such thing as experience.

Do not fool yourself, intelligence and knowledge are not the same thing. It is extremely easy to conflate the two and we're extremely biased to because the two typically strongly correlate. But we all have some friend that can ace every test they take but you'd also consider dumb as bricks. You'd be amazed at what we can do with just knowledge. Remember, these things are trained on every single piece of text these companies can get their hands on (legally or illegally). We're even talking about random hyper niche subreddits. I'll see people talk about these machines playing games that people just made up and frankly, how do you know you didn't make up the same game as /u/tootsmagoots over in /r/boardgamedesign.

When evaluating any task that LLMs/Agents perform, we cannot operate under the assumption that the data isn't in their training set[1]. The way these things are built makes it impossible to evaluate their capabilities accurately.

[0] before someone responds "there's no definition of intelligence", don't be stupid. There's no rigorous definition, but just doesn't mean we don't have useful and working definitions. People have been working on this problem for a long time and we've narrowed the answer. Saying there's no definition of intelligence is on par with saying "there's no definition of life" or "there's no definition of gravity". Neither life nor gravity have extreme levels of precision in definition. FFS we don't even know if the gravaton is real or not.

[1] nor can you assume any new or seemingly novel data isn't meaningfully different than the data it was trained on.

> [0] before someone responds "there's no definition of intelligence", don't be stupid.

Way to subdue discussion - complaining about replies before you get any.

But you're wrong, or rather it's irrelevant whether something has intelligence or not, if it is effectively diagnosing your illness from scans or hunting you with drones as you scuttle in and out of caves. It's good enough for purpose, whether it conforms to your academic definition of "having intelligence" or not.

Yeah, I mean, I don't know where all of this is going, but I do think that the ancients cared WAY more about "embodied knowledge" than we do, and I suspect we're about to find out a lot more about what that is and why it matters.