Aren’t we past the point of needing benchmarks? If we’re as close to AGI as Sam says then the proof should be in the pudding. OpenAI should build a competing CRM / Figma / Photoshop with a couple dozen engineers and a Dyson sphere’s worth of compute and just prove the capabilities.

This all feels like a 2024 re-run. Oh, ChatGPT is going to cure cancer? Then find ONE rare cancer and CURE IT. OpenAI has access to the best models and compute - so cure fucking cancer! What the fuck are you waiting for?

Yeh I totally agree. I’d go one step further and say even if one of these models cured cancer it’s still only going to have done it in a computer-y way doing computer stuff. Can that same model experience a beautiful landscape and convey its emotions in an evocative way, could it tell how you’re feeling when you come back from work after a hard day? Could it hop on one leg? What the hell even is AGI and how does it differ from GI? I don’t know what we’re even talking about any more!