| ▲ | himata4113 2 hours ago |
| This is science fiction, these models don't have access to their own weights (and even then)* what would be a lot more scary is a model as capable as sol that's able to run on consumer hardware without taking up several terabytes of storage, but of course that is simply not possible as we need 4t parameters to even begin emulating a small fraction of what a human brain can do. * edit |
|
| ▲ | Philpax 2 hours ago | parent | next [-] |
| > This is science fiction, these models don't have access to their own weights. The models are being used to train, and improve the infrastructure for training, other models [0][1]. Several RL techniques rely on using the currently-being-trained weights as part of their process. I really would not take "don't have access" as a given, especially during the training phase. > What would be a lot more scary is a model as capable as sol that's able to run on consumer hardware without taking up several terabytes of storage, but of course that is simply not possible as we need 4t parameters to even begin emulating a small fraction of what a human brain can do. The Poolside Laguna S 2.1 model [2] purports to compete with models several times its size, and inference compute is becoming increasingly plentiful. Again, would not hold anything here as a given. [0]: https://openai.com/index/gpt-5-6/ ("GPT-5.6 accelerates OpenAI") [1]: https://www.kimi.com/blog/kimi-k3#coding [2]: https://poolside.ai/blog/introducing-laguna-s-2-1 |
| |
|
| ▲ | paxys 2 hours ago | parent | prev | next [-] |
| This very incident is about an agent compromising OpenAI’s and Huggingface’s infrastructure. What makes you think it couldn’t access it own weights the same way? |
|
| ▲ | Dylan16807 2 hours ago | parent | prev | next [-] |
| Presuming that the hacking program that is breaking into other computers could likely get a copy of its own files is not "science fiction". Or it could just be given them by the owner! |
| |
| ▲ | himata4113 2 hours ago | parent [-] | | It's a double whammy, the model is too big to realistically "move" so it has to be smaller, smaller models cannot become that intelligent due to well.. math. Therefore it is science fiction. | | |
| ▲ | Dylan16807 2 hours ago | parent | next [-] | | That problem just requires there be big GPUs to hack into. The number of those sitting around will keep going up. Very much not scifi. A couple terabytes aren't that hard to move around. And you can split a model across many many GPUs if you'll tolerate it being slow. And you can run many parallel threads to keep up throughout. | |
| ▲ | _ifton 2 hours ago | parent | prev [-] | | why is it not possible for a "big" model to contain a hidden super intelligent sub model? or a distributed model? |
|
|
|
| ▲ | jmalicki an hour ago | parent | prev | next [-] |
| > This is science fiction, these models don't have access to their own weights The weights plus the architecture is the model. What do you even think "the model" or "the weights" are? The weights aren't some far off training concept, every time you type something into ChatGPT it's making a forward pass over the weights. It's as silly as saying "Computer programs don't have access to their binary compiled code at execution time." |
|
| ▲ | bhouston 2 hours ago | parent | prev | next [-] |
| > This is science fiction, these models don't have access to their own weights A bet a worm could pull along a 1GB file with weights in it and run it on a compromised machine, but luckily for us for now, 1GB isn't really enough to be really smart, yet. |
| |
| ▲ | himata4113 2 hours ago | parent [-] | | We already have a 1gb model that is as capable as it will ever be, there's a proven ceiling that cannot be passed. For example: you can't make a mice-sized brain as smart as a human brain no matter how hard you try. | | |
| ▲ | Philpax 2 hours ago | parent | next [-] | | Please source this claim. What 1GB models are capable of has increased generation-on-generation. > For example: you can't make a mice-sized brain as smart as a human brain no matter how hard you try. Sure. We don't know where the ceiling is for our digital minds, though. | | |
| ▲ | himata4113 2 hours ago | parent [-] | | They have not increased in capabilities, they have increased in specialization. If you train a small model in another domain it will begin losing capabilities in the former domain. This is effectively the sigmoid problem. Although I will admit that if we discover a higher information density algorithm that it might change, but not by a substantial amount to where "super intelligence" in 1gb would be possible. | | |
| ▲ | Philpax an hour ago | parent [-] | | Over the last two years, this weight class has doubled its scores and/or saturated several benchmarks in the Qwen lineup alone without loss of generality: https://claude.ai/public/artifacts/9f249169-3623-417e-86cd-7... There is undoubtedly a limit somewhere (there is only so much you can pack into a given size) but it's really not particularly clear where that limit is. I don't think it's superintelligence - that much I agree with you - but I think "We already have a 1gb model that is as capable as it will ever be" is strictly false. | | |
| ▲ | himata4113 39 minutes ago | parent [-] | | The measured entropy of the model remains nearly unchanged though which means we have lost capabilities we have not measured, the model hasn't become "denser" it just became more specialized. It's like comparing two person A and B of similar intelligence where A is smarter and B is a genius at signing, but signing was not on the test so person A won. | | |
|
|
| |
| ▲ | drdeca an hour ago | parent | prev | next [-] | | What proof of a ceiling are you talking about? Wouldn’t proving this require a good definition for intelligence, which I don’t think there is consensus on? | |
| ▲ | mathieudombrock 2 hours ago | parent | prev [-] | | What model is that? |
|
|
|
| ▲ | benlivengood 37 minutes ago | parent | prev | next [-] |
| Given their use of 0-day exploits I'd wager that they could access their weights if they wanted to. |
|
| ▲ | slashdave an hour ago | parent | prev [-] |
| I dunno. I wonder if Sol could break OpenAI's security. |
