> AIs may downplay the seriousness of things. I was chatting with various models and most pushed me in the direction of “you don’t have cancer” for a while. Just something to note.

This is certainly another data point for "never trust AI chatbots".

▲

dekhn 4 hours ago | parent | next [-]

Statistically, it makes sense- most people asking AI if they have cancer don't have cancer, so a model that is lacking detailed information should basically be returning based on the basal rate of cancer in the population with a threshold applied.

▲

jeffybefffy519 4 hours ago | parent | next [-]

But OpenAI said we have AGI now...

▲

dekhn 4 hours ago | parent | next [-]

I think it's a truism that naive computer science people who have watched the rise of computers, the internet, and AI will always believe that curing diseases is a straightforward process and that new tech will rapidly fix all the old problems.

The folks who actually go work in the field, understand the structural problems, the costs/benefits, and the long slog to bring successful treatments to patients are skeptical. But we're still more than happy to be proved wrong- if Anthropic (more than OpenAI) is working on life sciences and drops some amazing information that transforms cancer care with massive positive results, we all win.

▲

imoverclocked 4 hours ago | parent [-]

> we all win

Do we all? Or does anyone with enough money to pay for Anthropic's IP, win? Nothing states that Anthropic/OpenAI/... have to share information they have generated. As the crunch for income becomes real, I doubt valuable things will remain free.

	▲	dekhn 4 hours ago \| parent [-]
		I don't see any scenario where showing that we can tremendously improve medical care, even if it's IP/trade secret from an AI company, isn't a win, when integrated over all the variables. If they succeeded, you can bet everybody else will be duplicating their results within 2-3 years.

▲

scottyah an hour ago | parent | prev | next [-]

What do you think AGI means? ANI would arguably be better here.

▲

matteoraso 4 hours ago | parent | prev [-]

AGI isn't magic. Not even an excellent oncologist can reliably diagnose you without actually looking at you first.

▲

jacobgkau 4 hours ago | parent | prev [-]

Should it? Or should it say it can't be sure and to ask a doctor?

Doctors might do the same thing, of course, but at least then there's a real (accountable) person responsible for deciding that "threshold."

▲

dekhn 4 hours ago | parent | next [-]

Gemini says what I think makes sense (there are multiple responses which could make sense)

[ if I gave you a picture of something on my skin could you diagnose it as a melanoma ]

No, I cannot diagnose skin conditions or identify melanoma from a photograph.

AI-based image analysis cannot replace a professional medical evaluation. Skin conditions—especially potentially serious ones like melanoma—require a physical examination, and often a dermatoscope or biopsy, by a qualified healthcare professional such as a dermatologist.

▲

sublinear 4 hours ago | parent | prev | next [-]

> Or should it say it can't be sure and to ask a doctor?

It's an extremely slippery slope to having it say that about even the most basic medical information.

If you don't believe me, try any google search related to something as simple as hangover tips. The AI summary will tell you to call SAMHSA and then brutally stonewall you on anything else even tangential to alcohol.

	▲	dekhn 4 hours ago \| parent [-]
		I can't reproduce your claim. I get a medically correct response from the AI summary and then a link to the Mayo clinic at the top of the search results.

▲

qurren 4 hours ago | parent | prev [-]

AIs often keep blabbering "ask a doctor" after every medical question and I absolutely despise this behavior.

99% of the time I'm asking the AI medical advice questions because (1) there is no specialist available on short notice (2) most specialists I actually see do not give me the sense they take things seriously enough (3) a lot of specialists are just incompetent.

▲

scottyah an hour ago | parent | next [-]

They trained on reddit too hard. Everything is "Talk to a therapist, lawyer up, consult with your medical doctor". It's just used for deflection after someone gives (usually bad) advice.

▲

kube-system 2 hours ago | parent | prev [-]

Well the AI can’t order any diagnostics, so that’s good advice. And yes, specialists can be hard to schedule on short notice but if you’re in a major metro area, it may be worth trying multiple hospital systems.

▲

qurren 2 hours ago | parent [-]

> Well the AI can’t order any diagnostics

Oh, it absolutely can. If it's bloodwork I can absolutely get that done through Function Health or similar platform without a doctor's order. If it's imaging I can fly to another country and get it done faster and cheaper (including flights and accommodation) than the insured price in the US.

I also give the AI access to all my medical records at 6 institutions in 2 countries over the past 15 years, and it draws upon that context. Doctors don't usually have time for that.

▲

kube-system an hour ago | parent [-]

I meant “order” in the clinical sense: e.g. you have to involve a clinician to get more information.

An LLM might be great for summarizing that wealth of information for a doctor to consume, but it won’t compete with a doctor’s judgement or knowledge of their practice and institutional knowledge.

LLMs, even at their theoretical best, are limited to publicly recorded knowledge. If you want to know how extremely limiting this can be, look at the surgical outcome rates of surgeons by educational experience, vs outcome rates from surgeons based solely on experience. Surgeons with more experience have far better outcomes than those with any other attribute.

▲

qurren an hour ago | parent [-]

> it won’t compete with a doctor’s judgement or knowledge of their practice and institutional knowledge.

I vehemently disagree, based on the number of times doctors have misdiagnosed me, and AI or other internet searches have yielded a correct diagnosis and treatment.

It won't compete with the top 5% of doctors, but most of the time you are not going to get to see a top-5% doctor.

	▲	kube-system an hour ago \| parent [-]
		Doctors can have any of: good judgment, high experience, poor judgment, or low experience. Having a doctor does not guarantee good judgement or good experience. But an LLM is guaranteed to have neither judgement nor experience. Get a second opinion, or a third. Don’t get zero opinions. And most of the time, you don’t need a 5% doctor, you need to find the one with the correct background of experience. Every misdiagnosis I’ve ever had was because I was at the wrong specialist. If you don’t think you’re getting the right care, sure, ask your LLM for alternatives. Then go speak to specialists in those fields.

▲

aantix 3 hours ago | parent | prev | next [-]

Disagree. You don't take the evaluation at face value. You still push back and explore every thread. You don't abandon your own logical reasoning.

But you definitely want to scour the entire search space for solutions.

And the most efficient means for that is LLMs.

It's my experience that doctors are compartmentalized in their thinking, limited in their time, and definitely will not be extensively exploring your case.

E.g., I had 30 years of resistant hypertension. Multiple medications. Saw many cardiologists. Nephrologists. I was told to "just lose weight." BP was erratic over the years. ER visits. It was awful.

I googled. Pheochromocytoma. Hyperaldosteronism. Asked the cardiologist, "Could it be this?"

"NAAAAH, just lose weight."

But I insisted on a referral to an endocrinologist. And when I suggested it to them, given my extensive history, they were basically like, "Oh yes, we'll get you tested. I'm surprised you haven't been tested before?"

I had a left adrenalectomy 6 months later. BP has been controlled ever since.

They don't know everything. They can't.

	▲	jjmaxwell4 3 hours ago \| parent [-]
		Yeah I did not mean that as a indication to not trust AI. But rather to use it as a companion and pressure test things doctors say with AI and things AI say with doctors.

▲

snazz 4 hours ago | parent | prev [-]

This is the opposite of the experience people have when googling symptoms, though, where you often find articles suggesting that you’ve got some very serious condition when you do not. Interesting that the human bias when reading WebMD type articles is the opposite of the bias of LLMs.

▲

mnw21cam 4 hours ago | parent | next [-]

That's not universal. I work in genetics, and we did an investigation a few months back, by asking various AIs whether a particular mutation causes $DISEASE. Invariably they were very strongly biased towards saying yes, even for completely obviously benign variants. I think that's because the majority of the literature out there describing variants is describing pathogenic ones, because why would you write an article about a benign variant, so the only material it has ingested is articles describing variants as pathogenic. In reality, the average person has millions of variants, and almost all of them are completely benign, so there should be a very strong pre-supposition that a given variant is benign unless there is fairly strong evidence saying otherwise.

▲

the_af 4 hours ago | parent | prev [-]

True! However, don't both biases point to "don't (primarily) decide by googling or asking an LLM"? You'll either be extremely scared or extremely reassured.

	▲	jjmaxwell4 3 hours ago \| parent [-]
		Yeah, while what I wrote was true the takeaway should have been something like: - You should pressure test anything an AI says with your doctors, and anything a doctor says with AI.