My understanding is that it’s better than doctors themselves. But it’s probably the same as with autonomous driving: the bar isn’t just “be as good as humans”, it’s “be flawless”.

▲

haldujai an hour ago | parent | next [-]

It’s actually quite a lot worse than even doctors in training except for highly constrained experimental settings and a few very nice applications that are mostly too tedious/impractical for a human to do or are very basic detection tasks.

I am a radiologist and researcher predominately focused on AI.

▲

nesk_ 11 minutes ago | parent | next [-]

A friend of mine, a dermatologist, told me that LLMs are quite performant for melanoma analysis. Based on their own statistics, LLMs are able to beat humans with ~10 years of experience in the field.

They will never beat the human instinct tho, but they can be great tools sometimes. Unfortunately, LLMs mostly produce garbage.

▲

marcus_holmes an hour ago | parent | prev [-]

Thanks for the informed take :)

Do you think this will result in more routine/boring/tedious tests? Is the bottleneck on these things the human time to analyse them?

	▲	haldujai 17 minutes ago \| parent [-]
		I don’t think so, not beyond the current trend in medicine which is going up anyway. For some things, like 3D volume segmentation of structure or disease (e.g. CVA/stroke volume, cardiac muscle mass, iron quantification) the bottleneck is the time it takes so we currently use approximations like single longest dimension, circular regions of interest, etc. AI will dramatically increase accuracy allowing for more accurate treatment and easier large scale research with quantitative endpoints. Other things people think of like detection of aneurysms, fracture, lung nodules are not “hard” but AI has already added and will continue to add the second-reader benefit which will reduce detection errors. For this category the clinical benefit is as of yet unclear and we know that increased detection does not necessarily translate into improved patient outcomes and can in fact make them worse from over-diagnosis which means investigation related harms and over-treatment. We were already in a phase of “over detection” in much of radiology with advances in imaging technology so the incremental benefit of current AI remains to be seen and I personally think is going to be much more limited. I had a case recently where a 2 mm brain aneurysm was missed on 3 CT scans over 10 years but was picked up by AI so now is being followed annually. This is too small to treat considering the risks and a serious argument could be made that 10 years of stability is proof enough that this is almost certainly clinically irrelevant for this patient. Far more interesting areas of AI in imaging are in acquisition of acceleration (i.e. the medical equivalent of upscaling) which can dramatically decrease costs and increase accessibility as well as analyzing imperceptible features. It may not be a popular take here but in my opinion the future of radiology is like what we see in software engineering today - a skilled human equipped with AI will outperform humans without AI and AI without humans, the latter of which we are still several years away from prototyping due to various technical hurdles.

▲

goda90 an hour ago | parent | prev | next [-]

With these kinds of things, I want to see comparisons to trained, alert humans. Cut out all the distracted, stressed, tired, incompetent, intoxicated cases from the baseline. That includes rushed doctors at the end of a long shift.

A self driving car doing better than a drunk on the freeway doesn't reassure me that it'll do better than sober me in a snowstorm.

▲

BobbyTables2 an hour ago | parent | prev [-]

I’ve seen the same. But I don’t see that as a glowing beacon of progress.

A whole lot of doctors, if not most, didn’t pick their profession out of an interest in medicine…