> Even when a goal is simple and well-defined, sometimes it’s going to look much less achievable to experts than it actually is

I think it’s a compelling idea, that even if AI never achieves superintelligence, there might still be enormous value in their ability to relentlessly pursue a solution for much “longer” (relatively speaking) than a human. Even if they never let us pick the highest hanging fruit, if they instead let us pick every single low hanging fruit, that’s still a massive win.