| ▲ | NitpickLawyer 6 hours ago |
| > So they admit that they could do more, but they won't? Yeah, there's another paragraph: > Identifying and removing cyber-related data. During development, we observed vulnerability research performance improving steadily as post-training scaled. Tracing the source of this trend, we found that cyber-related content was embedded within the general mid-training corpus, and cyber-related tasks were present in the RL environment mix — a natural byproduct of training on broad data. In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness. We will continue to apply appropriate controls at every training stage, of which cyber-related data identification and restriction is one example. (emphasis mine) It's clear that cyber is a touchy subject right now. But it's also clear that RL works on it, because it's highly verifiable. So smaller models are going to become very interesting, because they can be fine-tuned later to cover this field. |
|
| ▲ | neuronexmachina 3 hours ago | parent | next [-] |
| I'm betting this is related to China's Announcement No. 48 last week about AI export controls, since releasing an open-weight model is effectively export: https://www.jdsupra.com/legalnews/china-proposes-new-export-... > Prohibited: high-performance detection technologies used to counter cross-border data transmission security risks, including deep-synthesis forgery detection (266401J); AI cyber offensive capability technologies, such as malicious code generation and mutation, automated mass generation and distributed large-scale deployment of anti-detection variants (266402J); large models trained using China’s Core Data (266501J); and high-performance AI agents designed for highly sensitive scenarios, such as critical information infrastructure, national defense and core government affairs (266502J). |
|
| ▲ | yorwba 5 hours ago | parent | prev | next [-] |
| They also show scores for a "Kimi model w/o cyber risk-reduction" https://statics.kimi.ai/kimi-blogs/kimi-k3-1/20261023/xdev-b... Presumably State Security will get access to that on request. |
|
| ▲ | dotancohen 3 hours ago | parent | prev | next [-] |
| > It's clear that cyber is a touchy subject right now.
And that's why we need models that understand "cyber" or application and data security. The offensive models are powerful, and they are in the hands of script kiddies now. The models writing our software and designing our cloud infrastructure had better understand these issues as well, just to avoid coding in vulnerabilities. |
|
| ▲ | fartfeatures 4 hours ago | parent | prev | next [-] |
| I think this is misguided. Considering how much code is AI written these days do we really want LLMs writing insecure code because we lobotomised them? |
| |
| ▲ | MattDaEskimo 4 hours ago | parent | next [-] | | I think it's political. Certain companies are exploiting as much as possible to push the boogeyman of AI for regulation. | | |
| ▲ | echelon 4 hours ago | parent [-] | | It's exactly this. If the "scary Chinese open weights models" can do "dangerous cyber", then we might see regulations spring up against open weights sooner than later. I hate Anthropic and OpenAI. As companies and leadership, anyway. Dario and Sam are 100% to blame for this. They're trying to turn us all into jobless slaves, sex workers, and Squid Game pawns. The only good future is open weights. They're trying their damnedest to keep us under boot. I hope the Chinese companies (and I say this as a patriotic American) slay these two companies and destroy their moats. In my mind, the two most dangerous people in the world are Dario and Sam. We need open weights to win. | | |
| ▲ | m348e912 3 hours ago | parent | next [-] | | > They're trying to turn us all into jobless slaves and sex workers. Speak for yourself, I don't think there is a market for many of us as the latter. | | | |
| ▲ | mebrainmush an hour ago | parent | prev | next [-] | | Please enjoy some time out in the nature, and chill out. These times are not ideal but what can we do other than just take it as it comes? All the hate and rage is not good for your health. | |
| ▲ | Hammershaft 3 hours ago | parent | prev | next [-] | | Why do you think Dario is dangerous? I think he's genuine on the dangers of AI capabilities moving faster than AI alignment. | | |
| ▲ | qlte 34 minutes ago | parent | next [-] | | Which is a great example of what makes him dangerous: with the exception of Sam Altman he's arguably the single person most responsible for the current capabilities arms race. Anthropic has pushed the frontier harder and faster than OpenAI or anyone else in the last few years. Beginning with Anthropic's founding to quickly raise huge amounts of cash with a pitch to investors to rapidly leapfrog ChatGPT in scale, and culminating with Mythos/Fable which carved out a whole new tier of super models. He's always talked big about Anthropic prioritising safety and their "Responsible Scaling" but his words just simply don't align with reality while they keep going bigger, faster, until only OpenAI could briefly and half-heartedly claim near parity at the top (while cutting corners in safety because the existential threat to their market position is far more pressing to any corporation than some abstract risk for humanity). Whether it's wilful cognitive dissonance, cynical opportunism, financial greed or something else, that kind of self-assured moral superiority paired with his real world record of self-interested decision making for Anthropic's market position is a big red flag. | |
| ▲ | smashed 2 hours ago | parent | prev [-] | | Because his solution is to block competition and capture all the market through regulation. | | |
| ▲ | sroussey 2 hours ago | parent [-] | | VCs will teach companies all about the wonders of regulatory capture! |
|
| |
| ▲ | felixgallo 2 hours ago | parent | prev | next [-] | | Sam, I'll grant you, is probably among the top 10 most dangerous, but Dario? The only thing he's done to piss people off is to predict, correctly, that releasing too-powerful models too-quickly would cause havoc and chaos. This prediction actively hurts his company! It actively harms his chances of a huge exit! It actively raises the probability that Anthropic and OpenAI get nationalized and treated in the same way as, for instance, the nuclear program! By comparison, do you think the Chinese models are going to be trained in such a manner that you enjoy the society that results? What makes you think that? | | |
| ▲ | echelon 2 hours ago | parent [-] | | > do you think the Chinese models are going to be trained in such a manner that you enjoy the society that results? I don't need government minders. I'll take the models and live free. |
| |
| ▲ | pixl97 3 hours ago | parent | prev [-] | | >I hope the Chinese companies (and I say this as a patriotic American) slay these two companies and destroy their moats. China cares about about stability way more than the US even, they aren't going to be big on AI models upsetting the governments power, and mass cyber attacks could do that. At the end of the day we are going to have to realize AI is both dangerous and governments are going to control it. Worse if you start making and releasing your own dangerous models the 4chan party van is going to visit you. | | |
| ▲ | echelon 3 hours ago | parent [-] | | > At the end of the day we are going to have to realize AI is both dangerous and governments are going to control it. Your thinking is dangerous. Do you work for Anthropic? AI is dangerous when these powers are being hobbled up by the select few. These systems threaten to put us all out of work. I refuse to be a slave to a company or a government that is taking away our livelihoods, scraping up our earnings potentials, then seeking to muzzle us and put us in jails with no rights. I would rather burn down all of tech than have AI be owned by the elites. Your thinking is a threat to all of our liberties. | | |
| ▲ | pixl97 2 hours ago | parent [-] | | >I would rather burn down all of tech than have AI be owned by the elites. Then you better get your torches out now. >I refuse to be a slave to a company or a government that is taking away our livelihoods Don't worry, governments are fervently working on AI drones to take away your life when you get really upset about this. Oh, and they are also working at AI to detect when you write all this online, and track you down then monitor you on things like flock cameras. As they say, you'll be the first with your back against the wall when the revolution comes, I likely will be beside you for being too subversive. >AI is dangerous when Incorrect, AI is just dangerous. It doesn't matter who has it. The more powerful it becomes the more dangerous it gets. So, as you can tell I am an AI doomer. ---- Now, don't get me wrong, I want freedom for me, I want freedom for other people. I do not believe the technologies we're developing are conductive to having any freedom at all. Hell, if we're lucky we'll be slaves instead of all of us dead, including the select few. >Your thinking is a threat to all of our liberties. Lol, now this is kind of funny. "I want freedom to think whatever I want, but you, you shouldn't think like that!". Careful or you'll max out your irony meter. Anyway, "My kind of thinking" is just next ____ prediction. I see what is happening in the world around us, and based on the shape of that probability I see what is going to happen. No, you and I are not getting unlimited AI to do whatever we want. Yes, governments are going to lock down the shit out of AI for end users. Yes, governments are going to declare advanced LLMs munitions. Balls in your court, better think of what your next move is. |
|
|
|
| |
| ▲ | 3 hours ago | parent | prev | next [-] | | [deleted] | |
| ▲ | zer00eyz 4 hours ago | parent | prev [-] | | Anthropic: Touch too much security stuff and we ban you, and keep your money. Kimi: We took out the worst of it so have at it for free. Everyone else: Back to my ablated model on openrouter for 1/10th the price where I can brute force issues. | | |
| ▲ | jacquesm 4 hours ago | parent | next [-] | | Ablation only works to remove stuff added in post training, if the training data does not contain the material then ablation will do nothing. | |
| ▲ | anonymousDan 4 hours ago | parent | prev [-] | | Which ablated model do you recommend for this? | | |
|
|
|
| ▲ | sheepscreek 5 hours ago | parent | prev | next [-] |
| It’s not much of a deterrent to a motivated organization. They are basically saying that models themselves are already capable. If you fine tune them with some cyber security tasks, that should make them potent. :-/ |
|
| ▲ | vessenes 5 hours ago | parent | prev [-] |
| Classic “rules for thee and not for me.” Agreed that this could be reinjected later. Makes me think of neo in the matrix: “I know zero days.” “Show me.” |
| |
| ▲ | zozbot234 5 hours ago | parent | next [-] | | What "rules"? They don't have to post-train the model on anything specific, it's their judgment call. | | |
| ▲ | vessenes 5 hours ago | parent | next [-] | | The "rule" here is, if you read the disclosure carefully, that they have cyber trained models in house. But you do not get them, you only get the non-cyber models. | |
| ▲ | mlmonkey 5 hours ago | parent | prev [-] | | Post-training _is_ a part of the whole training regime. |
| |
| ▲ | pixl97 3 hours ago | parent | prev | next [-] | | Governments also control who owns tanks and missiles. Do you think that going around making cyber weapons isn't going to get you buried in a shallow grave? | |
| ▲ | 4 hours ago | parent | prev [-] | | [deleted] |
|
