I think part of the problem is they’ve made it our problem.

Look I like Waymo. I think they’re neat and I trust them far more than any of the other companies. But in my mind being able to handle stuff like this is just a requirement to be on the roads in any non-trivial number. Like if they had two vehicles in this happened then OK that’s a problem but it was two vehicles in an entire city.

When you have enough on the road that you can randomly have six at one intersection you should absolutely be able to handle this by then.

I want them to do good. I want them to succeed. But just like airliners this is the kind of thing where people’s safety comes first.

What we saw happen looks like the safety of the Waymo and its passengers came above everyone else despite having no need to do that. There are certainly some situations where just staying put is the best decision.

The power went out and there are no other hazards on the road is not one of them. They made things worse for everyone else on average in a foreseeable situation where it was totally unnecessary. And that’s not OK with me.

This feels like the kind of thing that absolutely should’ve been tested extremely well by now. Before they were allowed to drive in large volumes.

▲

macintux 4 hours ago | parent [-]

Effectively they’ve turned any edge case into a potential city-wide problem and PR nightmare.

One driver doesn’t know how to handle a power outage? It’s not news. Hundreds of automated vehicles all experience the same failure? National news.

▲

scoofy 2 hours ago | parent | next [-]

I live in the affected neighborhood. There were hundreds of drivers that did not know how to handle a power outage... it was a minority of drivers, but it was a nontrivial, but nominally large number. I even saw a Muni bus blow through a blacked out intersection. The difference is the Waymos failed in a way that prevented potential injury, whereas the humans who failed, all fail in a way that would create potential injury.

I wish the Waymos handled it better, yes, but I think that the failure state they took is preferable to the alternative.

▲

Dylan16807 27 minutes ago | parent [-]

Locking down the roads creates a lot of potential injuries too.

And "don't blow through an intersection with dead lights" is super easy to program. That's not enough for me to forgive them of all that much misbehavior.

	▲	scoofy 10 minutes ago \| parent [-]
		> is super easy to program What?!? We’re talking about autonomous vehicles here.

▲

MBCook 4 hours ago | parent | prev | next [-]

Right. You know there are humans somewhere in the city who got confused or scared and mess up too. Maybe a young driver who is barely confident in the first place on a temporary permit, or just someone who doesn’t remember what you do and was already over-stressed.

Whatever, it happens.

This was a (totally unintentional) coordinated screw up causing problems all over as opposed to one small spot.

The scale makes all the difference.

▲

scottbez1 2 hours ago | parent | prev [-]

Yeah, the correlated risk with AVs is a pretty serious concern. And not just in emergencies where they can easily DDOS the roads, but even things like widespread weaknesses or edge cases in their perception models can cause really weird and disturbing outcomes.

Imagine a model that works real well for detecting cars and adults but routinely misses children; you could end up with cars that are 1/10th as deadly to adults but 2x as deadly to children. Yes, in this hypothetical it saves lives overall, but is it actually a societal good? In some ways yes, in some ways it should never be allowed on any roads at all. It’s one of the reasons aggregated metrics on safety are so important to scrutinize.