new | show | ask | jobs Github

rvz 2 hours ago

Nothing to see here and working as intended.

Imagine frontier abliterated models running at 2,400 TPS finding all these issues rapidly on web apps like this.

You are not ready security-wise when these models get faster. ("Formal verification" won't save you either)

▲

nicce an hour ago | parent | next [-]

Nothing saves if human is completely out of the loop

▲

swatcoder 2 hours ago | parent | prev [-]

Or just the natural consequence of adopting "move fast and break things" from top to bottom for 20 years, a house-of-cards methodology that was destined to fall in on itself whether or not such models ever arrived.

Formal verification doesn't solve all the problems, but alongside concerns like correctness, restrained surface area, supply chain awareness, treating PII as toxic waste to avoid rathee than treasure to collect can leave projects/orgs in a much better place than they let themselves get too.

Long way to go from here though.

▲

hnav 2 hours ago | parent [-]

This is one of the reasons anthropic nerfs their models. They want you to have an enterprise contract with some spend before they’ll allow you a couple of seats on cyber models, which you need since attackers are much more nimble with ablated open weights models.

	▲	neathack an hour ago \| parent \| next [-]
		As a side note to the "nerfing", I worked with a journalist on the analysis, and she wasn’t comfortable running `curl` on the command line. So she asked the AI to do it -- but it refused, citing concerns about cyber abuse. On the one hand, that’s kind of encouraging. On the other, it feels like the unskippable “don’t copy this movie” warning on a legally purchased DVD -- the one that gets removed from pirated versions. It seems to punish the wrong people.
	▲	tehjoker 2 hours ago \| parent \| prev [-]
		The ultimate capitalist model, create a never before experienced problem and sell the solution on a never ending subscription. Progress!