> They need to be around parity.

I don't think so.

The deaths from self-driving accidents will look _strange_ and _inhuman_ to most people. The negative PR from self-driving accidents will be much worse for every single fatal collision than a human driven fatality.

I think these things genuinely need to be significantly safer for society to be willing to tolerate the accidents that do happen. Maybe not a full order of magnitude safer, but I think it will need to be clearly safer than human drivers and not just at parity.

> negative PR from self-driving accidents will be much worse for every single fatal collision than a human driven fatality

We're speaking in hypotheticals about stuff that has already happened.

> I think these things genuinely need to be significantly safer for society to be willing to tolerate the accidents that do happen

I used to as well. And no doubt, some populations will take this view.

They won't have a stake in how self-driving cars are built and regulated. There is too much competition between U.S. states and China. Waymo was born in Arizona and is no growing up in California and Florida. Tesla is being shaped by Texas. The moment Tesla or BYD get their shit together, we'll probably see federal preëmption.

(Contrast this with AI, where local concerns around e.g. power and water demand attention. Highways, on the other hand, are federally owned. And D.C. exerting local pressure with one hand while holding highway funds in the other is long precedented.)

I know this sounds bad, but I wonder if you put an LLM in the vehicle that can control basic stuff (like the radio, climate controls, windows, change destination, maybe friendly chatter) but no actual vehicle control, people will humanize the car and be much more forgiving of mistakes. I feel pretty certain that they would..

> The deaths from self-driving accidents will look _strange_ and _inhuman_ to most people.

I like to quip that error-rate is not the same as error-shape. A lower rate isn't actually better if it means problems that "escape" our usual guardrails and backup plans and remedies.

You're right that some of it may just be a perception-issue, but IMO any "alien" pattern of failures indicates that there's a meta-problem we need to fix, either in the weird system or in the matrix of other systems around it. Predictability is a feature in and of itself.