| ▲ | dangom 5 hours ago |
| These are extremely jaw dropping results, but I feel even though it is easy to conflate them with RSI / intelligence explosion, another simpler explanation is that symbolic manipulation is something humans are notoriously bad at, hence we prizing individuals that can rise above and dedicate their lives to it, and call them intelligent. For computers though, symbolic manipulation is something they are extremely good at, and something that clearly scales well will memory and compute. Therefore I would argue that calling a computer superintelligent would, in analogy to do what we do with humans, require computers to rise above things that are extremely difficult for them. It is of course hard to judge what would be hard for them not being them for the introspection, but it seems the ability to operate outside the realm of symbolic or numeric manipulation would be precisely that. It stands to reason that if some proofs require chaining 250k pages together, no human would ever be able to put it together. That doesn't mean something that is capable of doing so is necessarily super-intelligent. Still very impressive, and if true, a wonderful result of the symbiotic interplay between a human and a machine, each at what they are good at. |
|
| ▲ | larsfaye 5 hours ago | parent | next [-] |
| Agreed. The fact is, Claude didn't get here on its own. It took someone who has been working with the RH since they were a student in college; some 40 odd years of experience. It's unfortunate we're always focused on humans getting usurped by an "AI", instead of the reality that these models only succeed when we're there to show them what success looks like. What if "superintelligence" is really a hybrid result of humans using and guiding these large pattern interpolators? |
| |
| ▲ | Lambdanaut 5 hours ago | parent | next [-] | | What if "anthrocentricity" is really humans being unwilling to acknowledge that they are made of the same vibrating condensed energy as everything else, such that other matter can just as easily be moved into an apparently self-determining pattern of behavior as we are. What if we're not the specialist snowflake? | | |
| ▲ | kridsdale1 4 hours ago | parent | next [-] | | I’m on board with this, and have been since I first had a hallucinogen. But the human brain for now still has 100x the parameter count as the largest models. And they don’t have any circuitry for all the “keep yourself alive” work our brains are doing all the time, which may be the majority. | |
| ▲ | slopinthebag 3 hours ago | parent | prev [-] | | how profound of you |
| |
| ▲ | jonahx 3 hours ago | parent | prev | next [-] | | > It's unfortunate we're always focused on humans getting usurped by an "AI", instead of the reality that these models only succeed when we're there to show them what success looks like. As with chess, the reign of the human-machine "centaur" as the greatest mathematician in the world will likely be brief. I'd guess within 5 years the humans won't be adding anything. I hope I'm wrong. | |
| ▲ | patrickkrusiec 4 hours ago | parent | prev [-] | | This is incorrect because you can always record the human prompts guiding the model and train it on them. This is how alot of progress in software was done and the success there shows that this generalizes beyond the specific prompts. You could probably prompt Fable 7 "Solve <famous hard math problem>" and it could do so without an expert mathematician guiding it. |
|
|
| ▲ | throwaway384034 2 hours ago | parent | prev | next [-] |
| It is impressive because of how large the search space is not entirely because of the level of abstract reasoning required to solve it. The reasoning trace produced that lead to this result could be a part of training that teaches the next version of Claude how to navigate incredibly hard problems making it more efficient. |
|
| ▲ | singularity2001 4 hours ago | parent | prev | next [-] |
| People suggested that language models are incapable of intelligence because they don't do symbolic manipulation. I think it's fair to call a vector with ten thousand numbers some kind of symbol, but then what's the state in your brain? |
| |
| ▲ | rcxdude 3 hours ago | parent | next [-] | | Yeah, I think it's not so much "math is symbolic manipulation so computers and by extension LLMs are going to be good at it". LLMs are not by construction so good at it because ML models are far more statistical approximators than formal logic systems. I think the main thing that math has going for it is sufficient formalisation through tools like Lean that it's very verifiable, and the latests hotness in improving models is RLVR which seems to really work very well when the verification is strong and relevant to the task (probably also why they're getting very good at writing code and especially finding exploits). (I would also say that computers are not good at math in the sense that LLMs are proving to be good at it: reliable symbolic manipulation is important for correctness but it's no use for actual progress in mathematics without some very good informal thinking/creativity as well) | |
| ▲ | layer8 3 hours ago | parent | prev [-] | | The reliable symbolic manipulation done here is in the form of Lean proofs, an external tool used by the LLM. One has to wonder how far an LLM alone would have gotten if formal proof systems didn’t exist they can verify their reasoning with. |
|
|
| ▲ | everyday7732 4 hours ago | parent | prev [-] |
| Can you point to a task which you think would be an indicator of RSI or an intelligence explosion? |
| |
| ▲ | dangom 2 hours ago | parent | next [-] | | I'm not sure I really buy into the way these concepts are currently defined. What progress to date was not RSI and did not build on top of earlier progress? The main characteristic of our physiology is that of adaption, and we have fairly grounded research on progressive overload as a means to guide adaption to whichever direction we want. Should we then call humans RSI machines if they go work out or read a book, or study to get better at the activities they carry out? Define an objective, choose an optimization algorithm, bring a loss down. If it keeps going down, do I call that RSI because the loss descended from the step that came before it? Now, I see the nuance that the optimization algorithm changes over time, leading to interesting higher order effects the likes of which Hoefstadter describes in the GEB, but fundamentally, the idea is the same. I also don't see what an intelligence explosion is supposed to mean. If we increase the models efficiency by 10x, and reduce costs by 10x, so we have 100x more software and math proofs being shipped by unit of time in say 2028, does that really solve all of our problems? Say that gets distributed over the world. Do we get every human to be computer literate?
Say that gets concentrated. Do we solve problems no one has the means to even care about? I would consider an intelligence explosion when the models tell us whether our efforts are well or misdirected as looking back from the hindsight of 10 years ahead, and capable of explaining why in simple English. | |
| ▲ | William_BB 3 hours ago | parent | prev [-] | | If you ask a layman, the answer would probably be a model that zero-shots any problem in any reasonable domain. E.g. zero shot Riemann hypothesis. No tricks like swarms of agents, no multiple agents correcting each other. No high, xhigh "thinking" levels. Simply provide an input, feed the input through the layers, answer out. Yeah, you can achieve something similar if you scale the compute infinitely and intelligently search the exponentially large search space. | | |
| ▲ | jonahx 3 hours ago | parent [-] | | > No tricks like swarms of agents, no multiple agents correcting each other. No high, xhigh "thinking" levels. Why would implementation matter? The thought experiment is that you have a black box that can do X. What is X that validates RSI/super-intelligence/etc? It's meant to clarify definitions and end the cycle of goal post moving. |
|
|