> no one else seems to find this to be quite damning for the AI services being offered, preferring instanced to be wowed by the convenience and speed at which they can be delivered unreviewed and unproven information

"Be wowed by the convenience and speed", or merely "take advantage of the mere availability"? What most people find to be damning about expert advice is that they simply can't get it anywhere, at any cost that they can afford.

▲

whatever1 2 hours ago | parent [-]

So if you want to do a surgery but you don’t see any surgeons around you ask a grocery butcher to have his way?

▲

sxg an hour ago | parent | next [-]

In certain circumstances, the answer is yes. If an airplane's pilots are incapacitated, do you simply give up and crash the plane because there are no other pilots on board? Or would you rather have someone on the ground try to coach a passenger into at least attempting to land the plane?

▲

close04 31 minutes ago | parent | next [-]

A passenger crashing the plane while trying to avoid a certain crash doesn’t make things any worse. An incompetent doctor trying to save you from certain death can make things so much worse. It’s all about weighing the best/worst outcome compared to where you are now.

	▲	microgpt 6 minutes ago \| parent [-]
		I hate to break it to you but death is certain for everyone. Emotionally realizing this and the complete inability to do anything about it is called an "existential crisis" and if you haven't had one or several yet, you will.

▲

frereubu 33 minutes ago | parent | prev | next [-]

That's an extreme edge case, which I don't think is in the context of the concerns in this thread.

	▲	sxg 19 minutes ago \| parent [-]
		The specific case doesn't matter--it's meant to make you think about the general question throughout this thread: when an expert isn't available, should non-experts use AI (or other tools) to help themselves? Sometimes the answer is yes because the potential benefits outweigh the potential harms (if any harms exist). But sometimes the answer is no because misleading/incorrect advice can cause a net harm.

▲

ChrisMarshallNY an hour ago | parent | prev | next [-]

As long as that passenger didn’t have the fish.

▲

jancsika 33 minutes ago | parent | prev [-]

You can choose a) a calm, level-headed passenger who knows they aren't a pilot, or b) a calm, level-headed passenger who almost has their pilots license but has a medical condition that prevents them from admitting when they lack certain knowledge.

Who do you choose to be coached by an expert on the ground?

	▲	rvnx 31 minutes ago \| parent [-]
		No thank you, I will ask Claude and then ask ChatGPT to challenge me, and do a couple of rounds like that. The first: Has no clue about anything and therefore no useful knowledge and cannot challenge me The second one: Is proven to willfully give wrong information and does even basic mistakes. The LLMs will do their best, even if imperfect, since they summarizes what appeared in books. I prefer to be grounded on what Airbus / Boeing manuals, or on what pilots training book said, than two far more unreliable sources.

▲

EA-3167 an hour ago | parent | prev [-]

People, especially in medical crises, are desperate for answers that they often can't get because their clinicians don't know. The illusion of an all-knowing guru who sounds like their doctor and tells them ANYTHING is extremely alluring. If you're waiting to hear back from a doctor about test results (which these days probably showed up on your online account the moment they were completed) can be agonizing.

Ok for pain in your shoulder it might not, but how about a woman with a lump in her breast waiting for the mammogram interpretation? How about someone trying to understand disturbing lab results? People are also often pushed these days to move through visits with doctors at a breakneck speed, but the AI will "hear you out" all day.

Part of this is a problem with the AI, part of it a problem with our healthcare systems, and part of it is simply human nature. If you think that OpenAI, Anthropic, Google and the rest weren't aware of this going in you must have very little faith in the intelligence of their members. It's not hard to imagine the future of LLM's should involve a hell of a lot of liability on the companies running it, but for now it's the Wild West.

▲

bilsbie an hour ago | parent [-]

> but how about

Whatever scenario you come up with my answer is the same.

As an adult I’d like to be able to choose what tools I use to learn about my condition regardless of how well it works or even if it’s likely to mislead me.

There’s risk in every aspect of life and we can’t baby proof everything.

	▲	baconmania 43 minutes ago \| parent \| next [-]
		>choose what tools I use to learn about my condition regardless of how well it works or even if it’s likely to mislead me. Even if it "works" so poorly that you're not actually learning about your condition?
	▲	EA-3167 38 minutes ago \| parent \| prev [-]
		If it's helping you learn about your condition then sure I agree. The issue here is that's not really the case, it's giving you the illusion that you're learning about your condition while feeding you hallucinations and half-truths at best. A recent look at medical advice from these things showed they're no better than a coin flip. So if you MUST have answers that are at most random guesses, I'd suggest saving a few bucks and asking a coin before flipping it.