Given the increasing use of them as agents rather than simple generators, I suggest a better analogy than "hammer" is "dog".

Here's some rules about dogs: https://en.wikipedia.org/wiki/Dangerous_Dogs_Act_1991

▲

skissane 4 hours ago | parent [-]

How many people do dogs kill each year, in circumstances nobody would justify?

How many people do frontier AI models kill each year, in circumstances nobody would justify?

The Pentagon has already received Claude's help in killing people, but the ethics and legality of those acts are disputed – when a dog kills a three year old, nobody is calling that a good thing or even the lesser evil.

	▲	ben_w 2 hours ago \| parent [-]
		> How many people do frontier AI models kill each year, in circumstances nobody would justify? Dunno, stats aren't recorded. But I can say there's wrongful death lawsuits naming some of the labs and their models. And there was that anecdote a while back about raw garlic infused olive oil botulism, a search for which reminded me about AI-generated mushroom "guides": https://news.ycombinator.com/item?id=40724714 Do you count death by self driving car in such stats? If someone takes medical advice and dies, is that reported like people who drive off an unsafe bridge when following google maps? But this is all danger by incompetence. The opposite, danger by competence, is where they enable people to become more dangerous than they otherwise would have been. A competent planner with no moral compass, you only find out how bad it can be when it's much too late. I don't think LLMs are that danger yet, even with METR timelines that's 3 years off. But I think it's best to aim for where the ball will be, rather than where it is. Then there's LLM-psychosis, which isn't on the competent-incompetent spectrum at all, and I have no idea if that affects people who weren't already prone to psychosis, or indeed if it's really just a moral panic hallucinated by the mileau.