| ▲ | exceptione 7 hours ago |
| Local models would be even better if they did not ship with all the refusal shenanigans built-in. You can safely bet organized crime has access to the best models without these hoops, which makes the case that the average user (=non-criminal) should have access too. As I understood from an ex-Anthropic employee, some orgs got access to Mythos based on their high enough spending level, not on other grounds. Either we are in command over the software, or the corp is in command over us via the software. I can on a theoretical level understand the concerns, but either we ban all LLMs or we have a level playing field for everybody. Let's not forget: defense and offense are different sides of the same coin in software. I guess this wouldn't apply to bio weapons, but I am not in the know about that. |
|
| ▲ | ninahaberl 7 hours ago | parent | next [-] |
| I’d expect these shenanigans to get much worse over time for the average Joe. Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug. Information has always been power and those who already have power won't just allow everyone else having the same tools as them |
| |
| ▲ | andsoitis 6 hours ago | parent [-] | | > Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug. It's an arms race. You have to run increasingly capable model partly because others can or do. |
|
|
| ▲ | dantudor 7 hours ago | parent | prev | next [-] |
| There are versions of Qwen3.8-27B that are unrestricted and available from hugging face. "It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails." |
| |
| ▲ | Aurornis 4 hours ago | parent | next [-] | | > There are versions of Qwen3.8-27B that are unrestricted and available from hugging face. The restrictions are not a single check in the model that can be removed. Those models on Huggingface are manipulated in different ways that also degrade the model’s intelligence. The degradation ranges from subtle to obviously broken, but it’s not free. When the restrictions are built into the model’s training sets you can try to alter the weights that are involved in the refusals, but that doesn’t mean that what’s left is useful or good knowledge for the same task. Those weights also might be involved in other tasks, so altering them can interfere with interactions that aren’t obviously related. | |
| ▲ | radlad 7 hours ago | parent | prev [-] | | > What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream — to remove the model's safety-alignment refusals. The result is a model that will comply with requests the original would refuse. Surely this has unintended side effects on output quality? | | |
| ▲ | andsoitis 7 hours ago | parent | next [-] | | > > What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream — to remove the model's safety-alignment refusals. The result is a model that will comply with requests the original would refuse. > Surely this has unintended side effects on output quality? Can you help me understand why that's the case? | | |
| ▲ | willy_k 5 hours ago | parent [-] | | Because deleting model weights after training is likely to cause knock-on effects in model knowledge and/or behavior. Targetting it might mitigate this but it’s a) not guaranteed that only censor-ey parameters get removed, and b) likely that removing those parameters still has effects on the effectiveness of related parameters. | | |
| ▲ | jszymborski 4 hours ago | parent [-] | | The weights aren't deleted, it's just additional fine tuning, is my understanding. |
|
| |
| ▲ | DiabloD3 7 hours ago | parent | prev | next [-] | | It does depending on the technique. | |
| ▲ | timmmmmmay 4 hours ago | parent | prev | next [-] | | Early attempts at this sort of thing definitely did, but these days the impact is minimal | |
| ▲ | miroljub 7 hours ago | parent | prev [-] | | A bit worse quality is a fine trade off when the alternative is no output (zero quality). | | |
|
|
|
| ▲ | TofuLover 7 hours ago | parent | prev | next [-] |
| Completely coincidentally, we're just about to launch a service that does exactly this (API access to uncensored open models)! We have a waitlist at the moment but will be live very soon! https://violentdelights.ai |
| |
| ▲ | 12345ieee 5 hours ago | parent | next [-] | | Regardless of the service, I'm amazed at the site's atmosphere, like it was a roleplaying server. | |
| ▲ | UI_at_80x24 6 hours ago | parent | prev | next [-] | | Given the context, your domain name is 'chefs kiss', perfect. I am completely curious what your legal defense would be though. "Come do things with AI that are probably illegal!" What?! We had no idea people would do things that are illegal! | | |
| ▲ | wongarsu 6 hours ago | parent | next [-] | | There is very little information that is illegal by itself. At least in the Western World, and especially in the US. The question is how far you get into the territory of aiding and abetting a crime But the reasonable defense is that the intended use cases are legal. The home page list a couple, and the 'writing fiction'/'helping authors' case alone covers almost everything. An author asking you how to best conduct a terrorist attack or how Meth is made are perfectly normal. Maybe even tame, compared to what some authors tend to research | |
| ▲ | mdp2021 5 hours ago | parent | prev | next [-] | | > defense Improper use is that of the user, not inherent to the tool. Scolio: guns. Respondeo: guns are much more specialized (one-use) than knives. Proper use of sharp knives when what was shipped was a butter knife is understandable. (The simile is not fully overlapping but should give the idea. The instrument must be flexible; if it is misused it is then a responsibility of the abuser.) | |
| ▲ | TofuLover 6 hours ago | parent | prev | next [-] | | I guess we'll burn that bridge when we get to it! | | |
| ▲ | TofuLover 6 hours ago | parent [-] | | More seriously though, I think we should be fine: we don't host any content, and what people do with the models is their own responsibility (legally speaking, in our jurisdiction, at least according to Claude -- we're talking to a real lawyer next week). Like any other provider, we offer no guarantees of sane, safe, or accurate results. | | |
| ▲ | gguingff 5 hours ago | parent | next [-] | | Thanks for bringing up a service like this, it's quite important. A few serious questions if you don't mind. Confidentiality? Do you use any sort of logging and if not do you have a way to guarantee that your hosting providers are not snooping? Price vs Vast or Runpod? If i have a very large or a very small workload do you have a competitive rate vs a gpu provider that offers private gpu access? Subscription vs Api costs? Do you only offer api rate or will you offer discounted tokens for subscription? Subscription friendly towards open source harnesses such as omp? Heretic ablation vs other methods? KL divergence scores? Do you post train the weights yourselves or do you offer weights trained by other organizations and is this information available on the service? Cache hit/miss pricing policy? 90/10 or a different cache pricing policy, and how long do conversions stay in kv cache? Quantized cache and model? Do you offer a choice if i want a quantized model for speed or a quantized cache? If not do you publish the information? SGlang vs vllm or other inference engine? Do you publish your engine stack details? Thank you kindly I find the competition in this space very lacking. | | |
| ▲ | TofuLover 4 hours ago | parent [-] | | These are great questions, thanks! I'll answer them in turn, in a list because I like lists. * Confidentiality: no logging, third party analytics, or anything like that. More details in our Privacy Poilicy [1]. Our hosting providers will have their own policies, but we're not running a super private service like Proton or similar. Might do some kind of secure tenancy in the future if there's demand. * Price: I think Runpod vs per-token are very different beasts and for different purposes. I really can't make a direct comparison, as it'll be based on use case, but we're going for convenience over price, so all else being equal I'd expect us to be more expensive for most users anyway (edit: i meant "than other API providers"! We'd definitely need to be cheaper or at least competitive with spinning up your own cloud infra. We'd have parallelism and economies of scale on our side for this). We have a lot of experience with running and optimising open models though, so that's part of the value proposition too. * Subscriptions: Only API for now. Maybe subscription later but honestly we prefer simplicity. My own experience with subscription plans is that they're usually sold at a huge loss at first, then the price creeps up as the service is enshittified. That feels like a bit of a scam to get users, and that's not really what we're about. We want to provide something specific, and aren't really concerned about scaling as fast as possible. Maybe we'll provide subscriptions if there's a real demand for it, but no plans at the moment to do so. * Methodology: we use abliterated models, but I've been advised to hold off talking about that for now. Might make a blog post about this though (when we have a blog). * Cache: yeah about 90/10 for pricing. We're still trying to find the sweet spot for tuning eviction. Running LRU with no guarantee/storage at the moment, could probably be less aggressive with retention, but that also has privacy surface area implications. Ongoing conversation. * Quantisation: my brother in christ, everyone runs quantised. :) We're initially targetting FP8 on most models, but have had great results with MXFP4 though. If we can pack more concurrency onto nodes without losing quality, we'll reflect that in pricing. Or we'll offer as a separate model for cheaper and give users the choice. Edit: I see you were asking specifically about speed, which MXFP4 doesn't improve, but maybe if there's demand we'll run other qaunts for speed increase, especially on the larger models. * Engine: vLLM gang all the way! For now at least, as it's what we have most experience with, and we find it the most flexible. We've been experimenting with SGLang though, and there's definitely some interesting optimisations we could do with it. Hope this answers your questions, at least the ones I could! The irony of that hasn't escaped me! [1]: https://violentdelights.ai/privacy |
| |
| ▲ | Xunjin 6 hours ago | parent | prev | next [-] | | >legally speaking, in our jurisdiction, at least according to Claude -- we're talking to a real lawyer next week That's going to be fun lol | | |
| ▲ | TofuLover 4 hours ago | parent [-] | | We look forward to providing many headaches to our lawyers going forward. |
| |
| ▲ | tiahura 4 hours ago | parent | prev [-] | | Let us know what your insurance is like. |
|
| |
| ▲ | weberer 5 hours ago | parent | prev [-] | | What can a LLM generate that's illegal in the USA? Specifically a text-only LLM? | | |
| ▲ | Zambyte 5 hours ago | parent | next [-] | | Blackmail, incitement, libel, and basically anything that comes up if you search "computer crimes". | |
| ▲ | xtracto 5 hours ago | parent | prev [-] | | Text for pamphlets criticizing the ICE? | | |
|
| |
| ▲ | laidoffamazon 3 hours ago | parent | prev [-] | | I really like this product idea but I really don’t want “violent delights” on my credit card that sounds extremely suspicious | | |
| ▲ | TofuLover 2 hours ago | parent [-] | | Er, yes, good point! We'll make sure billing is a bit more discreet than that! | | |
|
|
|
| ▲ | seanmcdirmid 6 hours ago | parent | prev | next [-] |
| Hugging face is filled with uncensored versions of your favorite local models, so in a way they are shipped without the refusal stuff, via the magic of fine tuning or however they get this stuff out of models. |
|
| ▲ | mdp2021 5 hours ago | parent | prev | next [-] |
| > with all the refusal shenanigans Given the faults in simulated Intelligence that LLMs have, and a comparatively low level - which means, lower judgement abilities - to the best of us, there is a strident match having such employee judge the intentions of the employer. Limiting the responses makes much more sense on cloud-based systems (you are using our infrastructure etc.). |
|
| ▲ | ramon156 7 hours ago | parent | prev | next [-] |
| heretics and manual iterations get you very far to the point where i have ethical questions about whether this should be possible |
|
| ▲ | binary132 7 hours ago | parent | prev [-] |
| Ehh, it’s at least given as the excuse for gain-of-function bioweapon research |
| |
| ▲ | datsci_est_2015 7 hours ago | parent [-] | | Digression, but this is the real Great Filter imo, not AI. I think technology advances to a point where it only takes one or two bad actors to type the right prompt to get a recipe for civilization-destroying bioweapons before you get anywhere near true AGI or anything relevant to the Kardashev scale. Biology is fragile. But not that that’s a good justification for hamstrung models. I think it’s just the inevitable endgame and it’s more sad than scary | | |
| ▲ | jeremyjh 7 hours ago | parent | next [-] | | I have the same concern. If it becomes possible to engineer Captain Trips with a budget in the low 8 digits it won’t really matter what else happens. | |
| ▲ | xyzzy123 6 hours ago | parent | prev [-] | | I don't fully understand the instinct to regulate local models for this? It seems like the wrong place to address the problem. You can download Ebola sequences right now if you want to. That's not the same as having an isolate. The difference is a lot of messy reality. This kind of work is not generally "one shot" (Claude make me a supervirus, make no mistakes), it requires lab space, iteration, and specific resources. It has a footprint. Wouldn't it make more sense to monitor / regulate facilities where you can sequence or request assembly of DNA, RNA, restrict and monitor the supply of key reagents and so on? | | |
| ▲ | datsci_est_2015 3 hours ago | parent | next [-] | | Yeah I was thinking more on the scale of nation-state level actors. Most leaders would not consider a bio weapon due to potential blowback on their own population, as they generally don’t want to be the leader of a nation of corpses and ashes. But if they don’t care, or if they already are the leader of a nation of corpses and ashes, maybe there’s not so much holding them back. In my layman’s perspective, bio weapons seem to be more dangerous and volatile than nuclear weapons. And increasingly easier to engineer as biotech advances. I think about the headlines of “microscopic robots that target tumors from within your body” and see the horrifying opposite side of the double-edged sword. | |
| ▲ | addaon 5 hours ago | parent | prev [-] | | It depends how easy it is (now, or in the near future) to turn information into weapons, and how realistic control of materials is. There's a reason we control access to plutonium, but information about metastable hafnium. | | |
| ▲ | xyzzy123 4 hours ago | parent [-] | | It does seem to me that for this specific problem the materials are a lot more amenable to control than the information is? There's also this weird revealed threat model thing going on? Like why does it make sense to support heavy LLM restrictions but leave benchtop oligo synthesisers completely unregulated? (Note: I do agree that wanting to regulate BOTH is at least a consistent and defensible position). I find it philosophically interesting because the problem is not strictly information control. Local models don't have any special extra information with respect to biological research. What has to be restricted is using information that's already publicly known in the wrong ways. |
|
|
|
|
