My previous job was at a startup doing BMI, for research. For the first time I had the chance to work with expensive neural signal measurement tools (mainly EEG for us, but some teams used fMRI). and quickly did I learn how absolute horrible the signal to noise ratio (SNR) was in this field.

And how it was almost impossible to reproduce many published and well cited result. It was both exciting and jarring to talk with the neuroscientist, because they ofc knew about this and knew how to read the papers but the one doing more funding/business side ofc didn't really spend much time putting emphasis on that.

One of the team presented a accepted paper that basically used Deep Learning (Attention) to predict images that a person was thinking of, from the fMRI signals. When I asked "but DL is proven to be able to find pattern even in random noise, so how can you be sure this is not just overfitting to artefact?" and there wasn't really any answer to that (or rather the publication didn't take that in to account, although that can be experimentally determined). Still, a month later I saw tech explore or some tech news writing an article about it, something like "AI can now read your brain" and the 1984 implications yada yada.

So this is indeed something probably most practitioners, masters and PhD, realize relatively early.

So now that someone says "you know mindfulness is proven to change your brainwaves?" I always add my story "yes, but the study was done with EEG, so I don't trust the scientific backing of it" (but anecdotally, it helps me)

There are lots of reliable science done using EEG and fMRI; I believe you learned the wrong lesson here. The important thing is to treat motion and physiological sources of noise as a first-order problem that must be taken very seriously and requires strict data quality inclusion criterion. As far as deep learning in fMRI/EEG, your response about overfitting is too sweepingly broad to apply to the entire field.

To put it succinctly, I think you have overfit your conclusions on the amount of data you have seen

But none of this (signal/noise ratio, etc) is related to the topic of the article, which claims that even with good signal, blood flow is not useful to determine brain activity.

The difference is that EEG can be used usefully in e.g. biofeedback training and the study of sleep phases, so there is in fact enough signal here for it to be broadly useful in some simple cases. It is not clear fMRI has enough signal for anything even as simple as these things though.

I'm not sure I understand. Wouldn't any prediction result above statistical random (in the image mind reading study) be significant? If the study was performed correctly I don't really need to know much about fMRI to tell whether it's an interesting result or not.

> When I asked "but DL is proven to be able to find pattern even in random noise, so how can you be sure this is not just overfitting to artefact?"

So here you say quite a mouthful. If you train it on a pattern it'll see that pattern everywhere - think about the early "Deep Dream" trippy-dogs-pictures nonsense that was pervasive about eight or nine years ago.

I repaired a couple of cameras for someone who was working with a large university hospital about 15 years ago, where they were using admittedly 2010s-era "Deep Learning" to analyse biopsy scans for signs of cancer. It worked brilliantly, at least with the training materials, incredible hit rate, not too terrible false positive rate (no biggie, you're just trying to decide if you want to investigate further), really low false negative rate (if there was cancer it would spot it, for sure, and you don't want to miss that).

But in real-world patient data it went completely mental. The sample data was real-world patient data, too, but on "uncontrolled" patients, it was detecting cancer all over the place. It also detected cancer in pictures of the Oncology department lino floor, it detected cancer in a picture of a guy's ID badge, it detected cancer in a closeup of my car tyre, and it detected cancer in a photo of a grey overcast sky.

Aw no. Now what?

Well, that's why I looked at the camera for them. They'd photographed the biopsies with one camera on site, from "real patients", but a lot of the "clear" biopsies were from other sites.

You're ahead of me now, aren't you?

The "Deep Learning" system had in fact trained itself on a speck of shit on the sensor of one of the cameras, the one used for most of the "has cancer" biopsies and most of the "real patient under test" biopsies. If that little blob of about a dozen slightly darker pixels was present, then it must be cancer because that's what the grown-ups told it. The actual picture content was largely irrelevant because the blob was consistent across all of them.

I'm not too keen on AI in healthcare, not as a definitive "go/no-go" test thing.

I think you're throwing the baby out with the bathwater, while also pointing to the missing pieces in our understanding of the brain and consciousness.

I also work in the field, specifically with sleep slow-wave enhancement.

Blood flow as a proxy for brain activity I always felt was a weak measure, as the brain activity involved across all manner of operating our biological systems, so is the increased blood flow measured in fMRI a response to cognition, or autonomic activity? What does that oxydation mean.

EEG is similarly flawed when we try to equate "brainwaves" to emotions and consciousness. I think we're almost better off measuring HRV, a much simpler measure, and more reliable.

I'm fascinated that so many people who discuss brainwaves think of them as actual "waves", when it is just how we plot electrical activity which creates a visual wave like pattern.

However, and this is specifically related to our work in sleep, we can detect slow-waves (I dislike that term, it's the synchronous firing of neurons) and we are able to stimulate this restorative brain function through sensory perception during sleep, and even create slow-waves in a lab using TMS.

Research linked on our website [1]

I agree the industry needs to stop projecting what we hope we're seeing with what is actually being measured, and we don't understand enough about how the brain works, but I think completely throwing away any brain related measures we have is going too far.

1 - https://affectablesleep.com/how-it-works#research

90% of papers I read in computer science / computer security speak of software written or AI models they trained that are nowhere to be found. Not on git nor via email to the authors.

Saw the same thing first hand with Pathology data. Image analysis is far more straightforward problem than fMRI, but sorry, I do not trust your AI model that matches our pathologist’s scoring with 98.5% accuracy. Our pathologists are literally guesstimating these numbers and can vary by like 10-20% just based on the phase of the moon, whether the pathologist ate lunch yet, what slides he looked at earlier that day…that’s not even accounting for inter-pathologist variation…

Also saw this irl with a particular NGS diagnostic. This model was initially 99% accurate, P.I. smelled BS, had the grad student crunch the numbers again, 96% accurate, published it, built a company around this product —-> boom, 2 years later it was retracted because the data was a lot of amplified noise, spurious hits, overfitting.

I don’t know jack compared to the average HN contributor, but even I can smell the BS from a mile away in some of these biomedical AI models. Peer review is broken for highly-interdisciplinary research like this.

There's fancier ML studies on EEG signal but probably not consistent enough for clinical work. For now, the one thing EEG can reliably tell is if you're having a seizure or not, if you're delirious (or in a coma) or not, or if you're asleep.

> but DL is proven to be able to find pattern even in random noise, so how can you be sure this is not just overfitting to artefact?

You test your DL decoder on held-out data. This is the common practice.