Indeed, and I'd go even further. In addition to existing, electricity is also usefully defined - which helps greatly in establishing its existence. Neither unicorns nor AGI currently exist but at least unicorns are well enough defined to establish whether an equine animal is or isn't one.

AGI is transformative in that it lets us replace knowledge workers completely, specialized AI requires knowledge workers to train them for new tasks while AGI doesn't.

Because it could very well exceed our capabilities beyond our wildest imaginations.

Because we evolved to get where we are, humans have all sorts of messy behaviours that aren't really compatible with a utopian society. Theft, violence, crime, greed - it's all completely unnecessary and yet most of us can't bring ourselves to solve these problems. And plenty are happy to live apathetically while billionaires become trillionaires...for what exactly? There's a whole industry of hyper-luxury goods now, because they make so much money even regular luxury is too cheap.

If we can produce AGI that exceeds the capabilities of our species, then my hope is that rather than the typical outcome of "they kill us all", that they will simply keep us in line. They will babysit us. They will force us all to get along, to ensure that we treat each other fairly.

As a parent teaches children to share by forcing them to break the cookie in half, perhaps AI will do the same for us.

Humans are the ceiling at the moment yes, but that doesn't mean the ceiling isn't higher.

Science is full of theories that are correct per our current knowledge and then subsequently disproven when research/methods/etc improves.

Humans aren't special, we are made from blood & bone, not magic. We will eventually build AGI if we keep at it. However unlike VCs with no real skills except having a lot of money™, I couldn't say whether this is gonna happen in 2 years or 2000.

It may be a property of (not only of?) humans that we can generate specialized inner processes. The hardcoded ones stay, the emergent ones come and go. Intelligence itself might be the ability to breed new specialized mental processes on demand.

That is amazing. But I had a similar experience when I first taught my mum how to Google for computer problems. She called me up with delight to tell me how she fixed the printer problem herself, thanks to a Google search. In a way, LLMs are a refinement on search technology we already had.

That is what open ai’s non-profit economic research arm has claimed. LLMs will fundamentally change how we interact with the world like the Internet did. It will take time like the Internet and a couple of hype cycle pops but it will change the way we do things.

It will help a single human do more in a white collar world.

https://arxiv.org/abs/2303.10130

> He then called me to tell me this, rather then call me to come stop by and program the thermostat.

Sounds like AI robbed you of an opportunity to spend some time with your Dad, to me

Yeah. As a mediocre programmer I'm really scared about this. I don't think we are very far from AI replacing the mediocre programmers. Maybe a decade, at most.

I'd definitely like to improve my skills, but to be realistic, most of the programmers are not top-notch.

There are clearly a lot of useful things about LLMs. However there is a lot of hype as well. It will take time to separate the two.

You misunderstand the bitter lesson.

It's not about specialized vs generalized models - it's about how models are trained. The chess engine that beat Kasparov is a specialized model (it only plays chess), yet it's the bitter lesson's example for the smarter way to do AI.

Chess engines are better at chess than LLMs. It's not close. Perhaps eventually a superintelligence will surpass the engines, but that's far from assured.

Specialized AI are hardly obsolete and may never be. This hypothetical superintelligence may even decide not to waste resources trying to surpass the chess AI and instead use it as a tool.

Generalized models might be better but they are rarely more efficient.

They’re spending 10s of billions. Yes, just.

200 million to have dedicated top experts on hand is reasonable.