Remix.run Logo
▲ FiberBundle 6 hours ago

When Atari games or Go were solved around 10 years ago, nobody was saying that superintelligence was within months just because reinforcement learning worked on a domain that allowed learning by self-play. Now we have a domain that is solvable by reinforcement learning with verifiable rewards. It's very impressive that these models are capable of doing the things they do, but the techniques aren't transferable to the vast majority of other domains. ML research also isn't verifiable in the sense that math is. People who claim it's verifiable, because there's a target that can directly be optimized don't understand why RL works for math. It's that lean is able to massively shrink the search space. This simply cannot be done in other domains in the same way.

▲bscphil 4 hours ago | parent | next [-]

Yeah, a lot of the RSI claims seem dependent on us having very specific knowledge about how much intelligence is required to build superintelligence that I haven't seen anyone claim to have. Maybe you have to be, I don't know, 100x smarter than humans before the recursive part of RSI becomes tractable. Or maybe there are fundamental limits you can do with the hardware that exists that we don't know about, which means building superintelligence will take more time. I'm not saying the 2027-2030 people are wrong, I just think it depends on a bunch of things we can't credibly claim to know at this point.

▲singularity2001 4 hours ago | parent | prev | next [-]

I'm not a fan of Dwarkesh, but even he laid out that machine learning research progress is very much verifiable. At least in the RSI sense, that our current approach can be drastically optimized.

▲hollowturtle 4 hours ago | parent [-]

I won't trust that source at all, it's all about hype and trust me bro considerations

▲munksbeer 2 hours ago | parent | prev | next [-]

The models are improving at certain fields, and one of those fields is creating new models, which are verified by applying them to fields where you can prove that they're better.

I agree they don't transfer to all domains, but we do not know how far the theoretical will overlap with practical science. It could be "not all that much" and we hit roadblocks quite soon, or it could be that as the models become better and better at reasoning and research within their theoretical domains, it turns out to translate really well to the real world.

We don't know, but my bet is that we will continue to see rapid progress and impacts on the real world, until we reach a point where sceptics will have to throw in the towel.

▲ 5 hours ago | parent | prev | next [-]
[deleted]
▲cubefox 5 hours ago | parent | prev | next [-]

LLMs don't produce Lean proofs directly, they produce the proofs in natural language, and only translate them afterward. The fact that we have Lean has nothing to do with LLMs being good at theorem proving.

▲gus_massa 4 hours ago | parent | next [-]

(I think we mostly agree.) From a previos post about Navier-Stokes, the English version may have small holes and the AI make a few changes and fixups when translating to Lean.

▲porridgeraisin 5 hours ago | parent | prev [-]

This is simply not true that they only invoke lean at the end. They invoke it multiple times in between to reduce the search space like GP said. See how e.g the prove2me harness (which anthropic used for FLT) works for more details.

▲killerstorm 5 hours ago | parent | prev [-]

What kind of a thing do you think AI won't be able to do in 2030? A specific thing please.

▲kridsdale1 4 hours ago | parent | next [-]

Love.

Throw a football at a wide receiver running in chaotic patterns while avoiding oncoming attackers.

Use chopsticks to pick up a single grain of rice.

Olympic level gymnastics via neural-control of muscle fibers in an extremely precise signaling orchestration.

You may say the above are robotics and not “the model can scale to this” domain but I’d counter an animal can do amazing things exactly because of its generalizable and plastic neural architecture and that is what’s in scope.

▲killerstorm 2 hours ago | parent | next [-]

Have you seen the progress in robotics lately? They aren't yet at a level you described, but quite close - so I'd say it would be possible in 3 years even without intelligence explosion. (E.g. if all frontier AI research stops we'll still have improvements in robotics.)

There are already Unitree robots which are much closer to Olympic level than an ordinary dude: https://www.youtube.com/watch?v=Y3zb4o47yqA You might argue but it's kinda at a level you described - I think if Unitree wanted to make a football-thrower they could just do it now.

▲drivebyhooting 3 hours ago | parent | prev | next [-]

Shoot I can’t do those things either!

▲senordevnyc 3 hours ago | parent | prev [-]

I’ll grant you love, and I’ll also grant you 2030 on these, especially the last one. I suspect the football and chopsticks ones are very achievable by 2030 if anyone cared enough to focus on them.

▲FiberBundle 3 hours ago | parent | prev | next [-]

Fully autonomously run a business that generates $10+ million in ARR.

▲matheusd 3 hours ago | parent [-]

What about one that generates $1+ million ARR?

▲FiberBundle 43 minutes ago | parent [-]

I find that unlikely as well. I should note though, that I incorrectly wrote "running a business", when I meant to say "scaling to $10+ million in ARR". I don't think there's necessarily that much difference in running a business of these two sizes, but scaling to $10m ARR is much harder than to $1m ARR.

▲zug_zug 4 hours ago | parent | prev | next [-]

Either AI won't be able to do this: - Make a version of itself 100x more efficient that runs on commodity hardware and undercuts athropic.

Or, alternatively, it will be able to do that. Either way there's no world where one company bottlenecks and holds the keys to AGI, and frankly that's good.

▲newsy-combi 4 hours ago | parent | prev [-]

- Keep electricity costs down

- Stop lying

- Design personal quantum computers

- Save fanatics from the delusion that AI will solve their life

▲killerstorm 4 hours ago | parent [-]

I mean something a human can do, but AI can't.

▲lijok 4 hours ago | parent | next [-]

Lay brick? Install pipe? Wire cable? Plaster wall?

▲pixl97 41 minutes ago | parent [-]

Here's a brick laying robot from 2017...

https://www.youtube.com/watch?v=6s17IAj-XpU

so yea....

▲newsy-combi 26 minutes ago | parent [-]

Try telling it to stop or move to a different wall in natural language. I mean, you might as well call a train AI with definitions this generous.

▲newsy-combi an hour ago | parent | prev [-]

Self-reproducing physical agents (robots with onboard AI and ability to change their surroundings/build physical things) with human or higher intelligence which don't need a global supply chain of microchips, electro motors, etc. but could live off the land. That's basically the human concept, right? Totally untenable for AI/robotics of the current generation. Would require something like 3D-printing new chips but also 3D-printing new printers. Probably never happening. AI is fundamentally different, reliant on data centers, which are immovable and more like organs than organisms. Robots only come out of factories, not out of other robots or eggs layed by robots. So as Jack of all trades in the game of survival on Earth, humans win for the foreseeable future. Also, AI currently can't exist without humans, whereas humans have thrived without AI. Whether a silicon based ecosystem can ever survive on its own is an interesting question in and of itself. Because no matter how smart data centers become, they degrade fast, and then what? A data center needs maintenance. But if robots maintain the data center, who will maintain the robots? Does this equation ever solve? Humans have bacteria, a complex microcosm of life maintaining them. I don't think such a dependency-hierarchy of "life forms" is feasible for AI/robots, and humans will always have to be in the loop. That's one answer to your question, and not just for the next 3 years.

What stands axiomatically are human-to-human connections that we're wired to crave. Not to mention human reproduction.

Other points: - Editing a live action movie that will get 8.0 or more on imdb (AI sucks at reading emotions and won't be able to tell apart good takes from bad ones) - Reliably making good jokes (many humans can do it, AI weirdly sucks at it) - Accountability (contradictory because of its undead nature) - Reliably not lying/cheating (with no accountability, incentives will always be misaligned)