Remix.run Logo
epolanski 19 hours ago

It's very interesting, because on one side we have clear data showing that autonomous cars at the Waymo level are safer than humans.

On the other hand we don't give technology the same pass we give to humans, meaning that we don't tolerate machine incidents the same way we tolerate human ones.

Which means that self-driving cars need to be a magnitude of order safer than humans.

And even though a self-driving car will never road rage or use the car as a weapon on christmas markets, we'll still tolerate humans driving more than one odd 1-in-a-billion ai-error caused incident.

overgard 19 hours ago | parent | next [-]

I don't think you can just look at the statistics though. What I mean is that, the main issue is that the category of mistakes they make are going to be inherently different from the category of mistakes that humans make. IE, humans are more likely to make emotional mistakes, but self driving is probably more likely to make perceptual mistakes or fail to respond to novel situations. I think the "fail to respond to novel situations" is probably the biggest blocker for people to trust them enough for widespread deployment. I'd love to see one of these things drive on a poorly mapped dirt road up in the mountains, for instance (I'm not saying it can't -- I just mean I would personally love to see how it can adapt to that kind of situation)

whack 19 hours ago | parent | next [-]

> I don't think you can just look at the statistics though. What I mean is that, the main issue is that the category of mistakes they make are going to be inherently different from the category of mistakes that humans make

Would you rather have a X% chance of a loved one dying from a human-category mistake? Or a 0.5X% chance of a loved one dying from a ML-category mistake? I would take the latter in a heartbeat. Why waste time optimizing for "categories" when you can optimize for your loved ones not dying

cortesoft 19 hours ago | parent | next [-]

While I agree with you, not everyone does.

You and I are thinking with a purely logical thought process, but most people don’t reason that way.

A lot of people want to be able to hold someone accountable, and that is more important than the pure probabilities.

They want to be able to blame a person for making a mistake.

johngalt 18 hours ago | parent | prev [-]

That is not the way human beings typically evaluate risks. 'Acceptable risks' are more about psychology and not a statistical calculation. A self driving car that is twice as safe as the human average would be considered an unacceptable risk.

Add something that is new/unknown + lack of control, and you will have to clear a very high threshold to be considered 'acceptably safe'. My guess is a minimum of a 0.1x.

SR2Z 17 hours ago | parent [-]

I don't think that human beings factor the rate of injury or death in at all, because human beings are notoriously bad with expected values from extreme conditions.

Once people get used to taking robotaxis, they won't care. The bar is "safe enough for very few people to have a bad experience." If you take 10 robotaxi rides in a row and they all go flawlessly your subjective interpretation of the risk will be that there isn't any.

I feel confident saying this because if you die in the US to anything other than old age, it was almost certainly a car crash. Despite this, most people don't think twice before getting in a car.

JumpCrisscross 19 hours ago | parent | prev | next [-]

> humans are more likely to make emotional mistakes, but self driving is probably more likely to make perceptual mistakes or fail to respond to novel situations

Humans fail chaotically. AVs, at least Waymos, fail safely. In urban environments, there is a case to be made within a few years for banning (or making it massively cumbersome) to manually pilot a private vehicle.

cucumber3732842 5 hours ago | parent | next [-]

Failing in ways that confuse the crap out of all the humans around (including pedestrians and cyclists) and demand their time and attention and thereby detract from the safety of the overall system is not "failing safely".

Nobody would make excuses for a rookie driver coming to a stop in a busy interchange. Only someone with the moral character of a career politician would attempt to construe that as anything other than detracting from overall safety. But a computer does it and it's fine?

stefan_ 18 hours ago | parent | prev | next [-]

That's an odd claim. Human failures are uncorrelated, but the Waymos all running the same software must be expected to fail in the same way, in the situations in which they fail. Which is why they often make the news; one Waymo stuck soon sees many of its brethren join it.

ericmay 18 hours ago | parent [-]

Don't humans fail in the same way too, broadly speaking? Wouldn't it depend on how you categorize failure?

If a Waymo failed because it missed a stop sign, and then another Waymo failed because it was confused about some strange markings on the road, is that not failing "the same way" in the same sense that a person failing to yield or driving drunk?

inigyou 18 hours ago | parent [-]

I think they're saying if one waymo misses a stop sign every waymo will miss the same stop sign

4fr2 18 hours ago | parent | prev [-]

[dead]

signatoremo 19 hours ago | parent | prev | next [-]

Statistics is still the most reliable indicator, because you can’t really tell if a situation is “novel”. How novel is novel? I doubt even Waymo knows for sure.

People will trust them by familiarity, not by any objective measures. Their trust grow as they see them more often, without bad accidents.

JLO64 19 hours ago | parent | prev | next [-]

My understanding is that they only operate in areas they’ve had approved and pre-mapped out for them though. Practically I don’t think the scenario you’ve described would ever happen, but I too would like to see it (I suspect they wouldn’t handle it well).

I live East of DTLA and twice I’ve seen people driving them around, presumably to map those areas out. It’s pretty annoying that I can’t use them for my late night commutes yet, but I’m eagerly looking forward to when I can!

madaxe_again 19 hours ago | parent | prev [-]

IDK, I've been in the car with someone when they just up and up drove off the road because they failed to notice that there was a corner - and that's a sober human. Another time, someone else, they rounded a corner, saw a tractor in the other lane, and were so surprised that they hit a tree. My mother has totalled six cars - she insists it was someone else's fault every time, but the statistics are somewhat damning. My wife's driving has given me haemmorhoids, I clench so fucking hard. Blind corners at twice the speed limit, looks over her shoulder to talk to passengers, and I constantly find myself buying wing mirrors, fenders, headlights, body panels, hubcaps, and anything else she can break off on traffic, rocks, or street furniture. Well, I did until someone finally took her license away from her for a year.

So... not so sure. Every AV I've ever been in has driven incredibly cautiously compared to a human driver.

jazzyjackson 19 hours ago | parent | prev | next [-]

> a self-driving car will never … use the car as a weapon

You are lacking in creativity and vision. With networked autonomous vehicles, all cars can be used as a weapon simultaneously.

hvb2 19 hours ago | parent | prev | next [-]

> meaning that we don't tolerate machine incidents the same way we tolerate human ones.

Any company that builds self driving cars or operates them should show how reliable they think they are. Not by showing pr material or stats, that's all 'free'.

Make yourself, as the company, legally liable. If you think it's good, put your name on it and own the risk.

When that happens, I'll trust whoever says it's safe.

bryan_w 10 hours ago | parent [-]

Why don't you think waymo is liable? Who else would be responsible for an accident in one of their cars?

hvb2 4 hours ago | parent [-]

It's more in general, with self/autonomous driving. In the waymo case the company owns the cars and all, which makes it more clear.

dylan604 19 hours ago | parent | prev | next [-]

When a human driver messes up, there's a person to hold responsible. When a robot car messes up, who is held responsible? If the human messes up to the point they are no longer allowed to drive, that person is no longer on the streets to cause more mistakes. When a robot car messes up, are all of the other cars pulled off the streets as well until the glitch is fixed? Every robot car is essentially the same driver.

cortesoft 19 hours ago | parent | next [-]

While we hold humans financially responsible for all accidents, we don’t hold them personally responsible in every case. Most accidents end with no arrests or even tickets. If your hand slips on the steering wheel, you don’t go to jail.

Robot cars would be the same. Insurance is financially responsible, and if the accident is deemed caused by a flaw, it would be the company that made them that is held responsible for fixing it.

Fogest 17 hours ago | parent | prev | next [-]

Don't we have automated systems all around the world with the same element at play? Elevators? Trains/Subways that are autonomous, robotic lawn mowers, self-driving little delivery vehicles on sidewalks, all kinds of automated machines in factories that kill people, etc...

I feel like you could come up with long lists of cases where there is no direct human to blame. Someone may be at fault down the chain, but nobody who was directly controlling the machine. Just like if a mechanic screws up my brakes and I end up crashing, it wasn't directly me who caused the crash, even though I was sitting in the seat.

phoghed 18 hours ago | parent | prev | next [-]

Why would it work any differently than any other category of product or service? If you’re rich you can sue them, if not you’re fucked.

Unless there’s a very bad and widespread issue. Then you join a class action and get $24 on a visa giftcard you can’t figure out how to redeem and then forget about.

elliottkember 19 hours ago | parent | prev | next [-]

Mistakes aren’t glitches when the environment is so dynamic. I imagine they are making errors constantly, and constantly learning from them.

When a robot car makes a mistake, all robot cars learn from the experience. When a human messes up, only that human learns from it.

epolanski 19 hours ago | parent | prev | next [-]

> When a robot car messes up, who is held responsible?

The company owning and operating the robot?

sudokatsu 18 hours ago | parent [-]

Ah yes, can’t wait to sue Google on equal monetary and political footing. It will be very easy, and very fair. /s

Synthetic7346 17 hours ago | parent [-]

Also how does one arrest a company? Put the c suite behind bars?

DonHopkins 18 hours ago | parent | prev [-]

I use Old Glory Robot Insurance.

https://www.youtube.com/watch?v=g4Gh_IcK8UM

munk-a 18 hours ago | parent | prev | next [-]

> or use the car as a weapon on christmas markets

On the contrary, autonomous vehicles (especially if something like automated package delivery becomes an offered service) allow car bombs without the suicide bomber inside. Instead you just place an explosive in the device and send it to the intended destination - as an organization the primary concern then becomes whether the ride is traceable back to you which might be do-able given how "frictionless" rideapps are as long as you get a burner phone, prepaid credit card and are very careful about making sure the device is clean and can't be tied to your person.

Right now the tech is limited to places where carbombings aren't commonplace, but I think driverless cars open a huge potential issue in other markets. I absolutely agree they're safer than drivers are, but let's not be hyperbolic.

JumpCrisscross 18 hours ago | parent [-]

> autonomous vehicles (especially if something like automated package delivery becomes an offered service) allow car bombs without the suicide bomber inside

Every city has courier services, including Uber.

inigyou 18 hours ago | parent [-]

An Uber driver was recently incarcerated for driving a package of drugs 7 hours.

JumpCrisscross 19 hours ago | parent | prev | next [-]

> we don't give technology the same pass we give to humans

This was the hypothesis. But based on e.g. FSD, it doesn’t seem borne out in reality.

decimalenough 19 hours ago | parent [-]

We have a much higher level of tolerance for things that benefit us while posing a risk to others ("should my own car have FSD") than we do for things that benefit others while posing a risk to us ("should Waymos be allowed in my neighborhood").

shakna 19 hours ago | parent | prev | next [-]

If a human makes a mistake, we give them the liability to own up to their mistakes, and carry the costs.

Most self-driving enterprises are trying very hard to ensure their non-driver continues to carry this liability instead of their company.

This helps a them v. us attitude. The company needs to be trusted widescale by the public, and do seem to undercut their own PR efforts on that, repeatedly.

brunoborges 19 hours ago | parent | prev | next [-]

> On the other hand we don't give technology the same pass we give to humans, meaning that we don't tolerate machine incidents the same way we tolerate human ones.

It comes down to accountability, IMO. We tend to be forgiven to humans because we are humans too and we know we, humans, make mistakes, and these can have serious consequences.

When a human makes a mistake, they are immediately accountable, and therefore judged and forgiven (or not). When technology makes a mistake, who's to blame? The CEO? The software developer who wrote the code? The tester? The architect?

Yes, we want technology to be an order of magnitude safer than humans for this reason, because we know that, given the current way our judicial system works, when technology makes a mistake, no one is held accountable.

sudokatsu 19 hours ago | parent | prev | next [-]

We don’t tolerate machine incidents because who is liable if an accident occurs? A company with a $4.6 trillion market cap? They’re not a legal opponent any of us really want to take on.

xnx 18 hours ago | parent | next [-]

Lawyers are salivating at the possibility of suing Waymo if they ever make a mistake. Much better to be hit by a Waymo (or a Tesla Robotaxi) than the typical uninsured, "judgement proof", human driver.

sgustard 18 hours ago | parent | next [-]

Exactly, imagine the huge sums of liability insurance Google is carrying to make payouts when the worst-case scenario happens. That financial burden is a prime driver of their operational safety record.

sudokatsu 18 hours ago | parent | prev [-]

Any stats to back up that the _typical_ driver is uninsured?

xnx 18 hours ago | parent [-]

An exaggeration on my part. About 16% of all drivers are uninsured (https://insurance-research.org/node/130), but "accident probability of uninsured drivers is 34 percentage points higher" (https://www.tandfonline.com/doi/full/10.1080/19439962.2018.1...).

sebzim4500 18 hours ago | parent | prev [-]

Good point because massive insurance companies can't afford lawyers and no one ever sues large companies.

panick21_ 3 hours ago | parent | prev | next [-]

The solution is that we should give humans and waymo way less tolerance.

andrewmutz 18 hours ago | parent | prev | next [-]

> On the other hand we don't give technology the same pass we give to humans, meaning that we don't tolerate machine incidents the same way we tolerate human ones.

I completely disagree with this. I don't "give a pass" to humans killing other humans with a car.

Every year that skeptics delay adoption of autonomous driving is another year where tens of thousands of deaths could be prevented.

munk-a 18 hours ago | parent [-]

Tesla, in particular, has given those skeptics so much ammunition to be mad at. The delay isn't purely due to luddites but also somewhat driven by prior obvious bad actions by self-driving car companies.

It's also likely that if a switchover is aggressively pursued it'll cause a massive social backlash and likely evolve into a culture war issue.

phinnaeus 18 hours ago | parent | prev [-]

> a magnitude of order

unless I'm missing something, you meant "order of magnitude"