Remix.run Logo
▲ ForHackernews 2 hours ago

This story is written like it vindicates AI medicine but really it seems like the human medical system dropped the ball on the MRI. ChatGPT didn't have any brilliant insight, it just (eventually) suggested the same thing a human doctor had already suggested, but this time the author and his wife pushed harder for it to happen.

To use the metaphor from the article, they found the keys in the last place they looked.

▲sa46 2 hours ago | parent | next [-]

> but this time the author and his wife pushed harder for it to happen

This is the essence of how to make the medical system work for you. Advocate for yourself. AI helps know what to ask, why you should ask, and how to get past the first few layers of dismissals.

▲quantumwoke 2 hours ago | parent | next [-]

In this case it took roughly four years for AI to suggest a test. How should we interpret the utility of AI in this situation?

▲ryanisnan an hour ago | parent [-]

You don't understand. They were only able to feed in information that they obtained with their human medical team. So the AI was heavily constrained in what it knew.

▲quantumwoke an hour ago | parent [-]

AI was able to suggest a test for four years and it didn't, and I'm wondering why it didn't.

▲ForHackernews 2 hours ago | parent | prev [-]

Again, this seems more like an indictment of the American medical system than kudos for AI.

For the developers here, consider: "So our CI pipeline is super flaky, and it you have to restart it in a variety of non-intuitive ways. LLMs have been a massive breakthrough because the bot can run in an endless loop reading the confusing error messages and retrying the pipeline with different environment variables until it passes!"

▲40four 2 hours ago | parent | prev | next [-]

The human medical system drops the ball all the time unfortunately. What I don’t understand is how they didn’t see that giant fibroid from ultrasound alone. An MRI shouldn’t have been needed. Then the doctor trying to talk them out of the MRI, I hope they were embarrassed after. So upsetting. My wife’s friend had something similar happen after 2-3 years of unsuccessful fertility treatment, then they finally realized she had endometriosis. Seems like that should have been looked into sooner.

▲margalabargala 2 hours ago | parent [-]

> What I don’t understand is how they didn’t see that giant fibroid from ultrasound alone. An MRI shouldn’t have been needed

This is really kinda the crux of it.

The AI suggested the MRI not because it had some insight into "oh there's something only an MRI can detect going on", it was just a "collect more data" situation.

Frankly, the author of the blog should probably have gotten a second opinion from a different fertility doctor sometime in the intervening time period.

As someone who also went through multiple rounds of IVF with my partner, the number of ultrasounds you get during the process is immense. It's hard to call missing a peach-sized fibroid that many times anything other than gross incompetence.

▲quantumwoke an hour ago | parent [-]

I think this is where there is simply not enough information in the post. Fibroids can change in size over time. That being said, there are multiple types of fibroids apparently so it may be a type that wouldn't show up on ultrasound.

▲1shooner an hour ago | parent | prev | next [-]

>it just (eventually) suggested the same thing a human doctor had already suggested

Of course, but what else would you think an LLM does? That is the insight: sifting through a sea of information and parsing it without the same emotional or other cognitive biases the original human recipient applied to it.

▲Den_VR 2 hours ago | parent | prev | next [-]

Doctors don’t need brilliant insight, they need to essentially leverage human medical knowledge against unwellness in human beings. This is confounded by two measures: too many patients per doctor, and too much human medical knowledge.

▲saulpw 2 hours ago | parent | prev [-]

I mean if "AI medicine via chatbot" gets better results than the human medical system, I'd say that vindicates it..

▲peesem 2 hours ago | parent [-]

this is one case. there's no proof here that AI will generally do better than human doctors. consider also that the power it has to convince people to push for better medical treatments can also be used to push for worse ones, or to halt standard treatment entirely, at which point it is much harder for the medical system to intervene as they aren't interacting with doctors.

▲nerevarthelame an hour ago | parent | next [-]

It feels like moving toward a local maxima. In this example the LLM advice outperformed one doctor who dropped the ball in some ways, but I do not look forward to a world where we make it even more difficult to talk to doctors, and route most medical interactions to LLMs. I expect that in 5 years I'll be futilely shouting "representative" into my phone as my appendix ruptures.

With this sort of advice from LLMs, I think there's a lot of selective recall that's easily overlooked -- where we gloss over all the bad, dead-end lines of advice we seek from the LLM oracle.

▲bradknowles 2 hours ago | parent | prev [-]

There is some evidence that has been collected so far that, taken across large sample sizes, good AI systems are now better at diagnosing certain diseases, etc…. We started here with Mycin and e-Mycin back in the 60s. Hopefully, these systems can now start to be better integrated into modern medicine systems.

However, the real problem is that modern AI systems still hallucinate. And there is not yet any evidence that I’ve seen that AI systems can be trusted to work at these levels of efficiency and correctness at the tactical single-patient level, as opposed to the strategic population level.

I think lots more work needs to be done here.