| ▲ | comex 4 hours ago |
| I'm skeptical that this particular task would get blocked, though I could be wrong. But I agree that the moat is a problem in the general case. |
|
| ▲ | A_D_E_P_T 4 hours ago | parent | next [-] |
| I don't know about cryptography, but if you try anything in organic chemistry or biology, you virtually always get flagged and downgraded. As this is going on, Anthropic has opened up their own biology lab. > https://www.reuters.com/world/anthropic-quietly-sets-up-biol... There is very much a situation where people in frontier labs have access to exceptional capabilities that are being kept from the rest of us, and they're using it to strip-mine technical and scientific problems. |
| |
| ▲ | sillysaurusx 3 hours ago | parent | next [-] | | I ran into this situation and was able to work around it by running a jailbroken Chinese model that turned out to be quite capable. I think it was Qwen. | |
| ▲ | derwiki 4 hours ago | parent | prev | next [-] | | Strip mining is a great phrase here, thank you | | | |
| ▲ | alwillis 4 hours ago | parent | prev [-] | | > There is very much a situation where people in frontier labs have access to exceptional capabilities that are being kept from the rest of us, and they're using it to strip-mine technical and scientific problems. If these capabilities were available to the public, someone somewhere would make a biological weapon or create a virus we've never seen before that could kill many millions of people. Stanford researchers have demonstrated it’s possible [1]. [1]: "AI creates 16 new viruses from scratch, showing promise for drug resistance and drawing warnings about potential for misuse" - https://www.cnn.com/2026/08/06/health/ai-viruses-bacteriopha... | | |
| ▲ | hgoel 2 hours ago | parent | next [-] | | Biologists already addressed this idiocy a few weeks ago [1]. Making a biological weapon requires facilities that only some nation states - and comparable companies- have access to. AI isn't going to magically assemble a DNA sequence for you and a computer design is not a guarantee of something that works. AI restrictionists love to pretend that their toys will, with the power of intelligence, magically bypass fundamental limitations of reality to conjure up fantastical weapons of mass destruction if they throw enough computers at the task. Unfortunately for them, that is not how the world works. [1] https://www.businessinsider.com/researchers-ai-apocalypse-sk... | |
| ▲ | girvo 3 hours ago | parent | prev [-] | | We already have controls for that: we control access to the reagents and equipment needed. Yknow, like we always have. Information is not dangerous. This is a ridiculous argument to me, but maybe my chemistry degree has me biased. | | |
| ▲ | jiggawatts 2 hours ago | parent [-] | | Not to mention that making an actual novel virus requires thousands of dollars of equipment, serious lab skills, and even then is so dangerous that the wannabe terrorist is more likely to infect and kill themselves than cause mass causalities. It's the equivalent of making pipe bombs at home. The instructions are out in the public, but that's not a problem because the idiots that make pipe bombs blow themselves up more often than they succeed in doing terrorism. |
|
|
|
|
| ▲ | jcomp 4 hours ago | parent | prev | next [-] |
| Broadly, the downgrading is still way too heavy-handed and clumsy. As of yesterday “How easy to crack is WPA2” downgraded to opus 4.8. Granted I could probably have phrased it differently and gotten a response, but the point is I shouldn’t have to. It can’t simultaneously be smart enough to reverse the ECDSA curve seed and not smart enough not to destroy the world by answering a trivial question with a scary word in it. |
|
| ▲ | akersten 4 hours ago | parent | prev | next [-] |
| At minimum we don't have access to "Claude thoughts" like the ones the article highlighted, right? That's part of the hidden reasoning? |
|
| ▲ | petesergeant 4 hours ago | parent | prev | next [-] |
| Fable 5.1 has downgraded me in the past week alone for the crimes of describing a coordinating server as "the C&C server" ([cyber]), for doing some statistical analysis on some toy Vigenère-ciphered text ([cyber]), and for asking it to help me find patterns in some semi-random text ([bio]), so it is fair to say I am not skeptical. |
|
| ▲ | mike_d 4 hours ago | parent | prev | next [-] |
| I have access to Claude's "defensive" cyber model and it appears to take no issue with the prompt, but I suspect that is because it uses today's publication as the basis for an answer. In the past my attempts to work on crypto related problems have been met with needing the higher tier of "offensive" access. |
|
| ▲ | richwater 4 hours ago | parent | prev [-] |
| It doesn't matter what is blocked today, it matters what is blocked tomorrow. If we allow these providers to promote regulatory capture they become the arbiter of what anyone is "allowed" to do |
| |
| ▲ | ragequittah 2 hours ago | parent [-] | | I think when the open source models catch up (likely soon) we'll long for the days when companies could control it like this. I could easily see a teenager with openclaw and a single prompt causing millions of dollars in damage. Multiply that by the number of stupid people in the world and we have a problem. |
|
