If the new AGI benchmark is "be Einstein/Feynman" then we've hit AGI.

jmalicki 27 minutes ago | parent | next [-]

What if it can be Einstein, but can't draw a Pelican, write a solid college-level essay, or fold clothes?

The ability to do a ton of book learning in training, and pull in tons of related context at once, is superhuman in some ways, but lags a lot in others.

▲

block_dagger 36 minutes ago | parent | prev [-]

Wouldn’t that mean producing novel work like relativity and QED?

	▲	jmalicki 9 minutes ago \| parent [-]
		I would maybe argue that Einstein was the most LLM-like of great thinkers. A lot of his great discoveries were mostly that he was very knowledgeable about the bleeding edge research in a number of disparate areas, and was able to have the aha moment where he could make the connections for how to integrate them. A lot of other thinkers who created new fields from scratch are probably way harder for an LLM to crack. That is very aligned with an LLMs ability to have superhuman knowledge in wide areas.