It's so much worse than this.

For task fMRI, the test-retest reliability is so poor it should probably be considered useless or bordering on pseudoscience, except for in some very limited cases like activation of the visual and/or auditory and/or motor cortex with certain kinds of clear stimuli. For resting-state fMRI (rs-fMRI), the reliabilities are a bit better, but also still generally extremely poor [1-3].

There are also two IMO major and devastating theoretical concerns re fMRI that IMO make the whole thing border on nonsense. One is the assumed relation between the BOLD signal and "activation", and two is the extremely horrible temporal resolution of fMRI.

It is typically assumed that the BOLD response (increased oxygen uptake) (1) corresponds to greater metabolic activity, and (2) increased metabolic activity corresponds to "activation" of those tissues. This trades dubiously on the meaning of "activation", often assuming "activation = excitatory", when we know in fact much metabolic activity is inhibitory. fMRI cannot distinguish between these things.

There are other deeper issues, in that it is not even clear to what extent the BOLD signal is from neurons at all (could be glia), and it is possible the BOLD signal must be interpreted differently in different brain regions, and that the usual analyses looking for a "spike" in BOLD activity are basically nonsense, since BOLD activity isn't even related to this at all, but rather the local field potential, instead. All this is reviewed in [4].

Re: temporal resolution, essentially, if you pay attention to what is going on in your mind, you know that a LOT of thought can happen in just 0.5 seconds (think of when you have a flash of insight that unifies a bunch of ideas). Or think of how quickly processing must be happening in order for us to process a movie or animation sequence where there are up to e.g. 10 cuts / shots within a single second. There is also just biological evidence that neurons take only milliseconds to spike, and that a sequence of spikes (well under 100ms) can convey meaningful information.

However, the lowest temporal resolutions (repetition times) in fMRI are only around 0.7 seconds. IMO this means that the ONLY way to analyze fMRI that makes sense is to see it as an emergent phenomenon that may be correlated with certain kinds of long-term activity reflecting cyclical BOLD patterns / low-frequency patterns of the BOLD response. I.e. rs-fMRI is the only fMRI that has ever made much sense a priori. The solution to this is maybe to combine EEG (extremely high temporal resolution, clear use in monitoring realtime brain changes like meditative states and in biofeedback training) with fMRI, as in e.g. [5]. But, it may still well be just the case fMRI remains mostly useless.

[1] Elliott, M. L., Knodt, A. R., Ireland, D., Morris, M. L., Poulton, R., Ramrakha, S., Sison, M. L., Moffitt, T. E., Caspi, A., & Hariri, A. R. (2020). What Is the Test-Retest Reliability of Common Task-Functional MRI Measures? New Empirical Evidence and a Meta-Analysis. Psychological Science, 31(7), 792–806. https://doi.org/10.1177/0956797620916786

[2] Herting, M. M., Gautam, P., Chen, Z., Mezher, A., & Vetter, N. C. (2018). Test-retest reliability of longitudinal task-based fMRI: Implications for developmental studies. Developmental Cognitive Neuroscience, 33, 17–26. https://doi.org/10.1016/j.dcn.2017.07.001

[3] Termenon, M., Jaillard, A., Delon-Martin, C., & Achard, S. (2016). Reliability of graph analysis of resting state fMRI using test-retest dataset from the Human Connectome Project. NeuroImage, 142, 172–187. https://doi.org/10.1016/j.neuroimage.2016.05.062

[4] Ekstrom, A. (2010). How and when the fMRI BOLD signal relates to underlying neural activity: The danger in dissociation. Brain Research Reviews, 62(2), 233–244. https://doi.org/10.1016/j.brainresrev.2009.12.004, https://scholar.google.ca/scholar?cluster=642045057386053841...

[5] Ahmad, R. F., Malik, A. S., Kamel, N., Reza, F., & Abdullah, J. M. (2016). Simultaneous EEG-fMRI for working memory of the human brain. Australasian Physical & Engineering Sciences in Medicine, 39(2), 363–378. https://doi.org/10.1007/s13246-016-0438-x

▲

freehorse 6 days ago | parent | next [-]

Re: temporal resolution

Even if neuronal activity is (obviously) faster, the (assumed) neuro-vascular coupling is slower. Typically there are several seconds till you get a BOLD response after a stimulus or task, and this has nothing to do with fMRI sampling rate (fNIRS can have much faster sampling rate, but the BOLD response it measures is equally slow, too). Think of it as that neuronal spiking happens in a range of up to some hundred milliseconds and the body changing the blood flow happens much slower than that.

The issue is that measuring the BOLD response, even in best case scenario, is a very very indirect measure of neuronal activity. This is typically lost when people referring to fMRI studies as discovering "mental representations" in the brain and other non-sense, but here we are. Criticising the validity of the BOLD response itself, though, is certainly interesting.

▲

D-Machine 6 days ago | parent [-]

Right, my point is sort of that both the BOLD response and fMRI sampling rates are far too "slow" (not nearly approaching the Nyquist frequency, I guess) a priori to deeply investigate something as fast as cognition.

▲

kspacewalk2 6 days ago | parent | next [-]

Depends on what you mean by cognition, but as you yourself said, BOLD may be correlated with certain kinds of long(er)-term activity, and that in itself is very useful if interpreted carefully. No one claims to detect single "thoughts" or anything of the sort, at least I haven't seen anything so shameless.

	▲	D-Machine 6 days ago \| parent [-]
		Well, a lot of task fMRI designs are pretty shameless and clearly haven't taken the temporal resolution issues seriously, at least when it comes to interpreting their findings in discussions (i.e. claiming that certain regions being involved must mean certain kind of cognition, e.g. "thoughts" must be involved too). And there have definitely been a few papers trying to show they can e.g. reconstruct the image ("thought") in a person's mind from the fMRI signal. But I don't think we are really disagreeing on anything major here. I do think there is likely some useful potential locked away in carefully designed resting-state fMRI studies, probably especially for certain chronic and/or persistent systemic cognitive things like e.g. ADHD, autism, or, perhaps more fruitfully, it might just help with more basic understanding of things like sleep. But, I also won't be holding my breath for anything major coming out of fMRI anytime soon.

▲

ryandv 6 days ago | parent | prev | next [-]

> BOLD response and fMRI sampling rates

Funny, because these exact measures [0] were brought up in response to a similar claim I made over a year ago [1] about the resolution of our instrumentation.

There would appear to be a worrying trend of faith in scientism, or the belief that we already have all the answers squirreled away in a journal somewhere.

[0] https://news.ycombinator.com/item?id=41834346

[1] https://news.ycombinator.com/item?id=41807867

▲

D-Machine 6 days ago | parent [-]

It's a bit funny, the qualia thing and sampling rates.

Obviously we hope what we learn from e.g. psychology and fMRI will help us explain more things about the mind, and surely most researchers in psychology hope their research will help us get some answers on things related to qualia as well. And almost certainly most good / consistent reductionist researchers must believe that qualia arise from the brain, at least in significant part.

However, precisely by this reductionist logic, and since it is immediately and phenomenally clear that the rate of change of qualia in the mind (or the "amount" of different qualia, i.e. images or sounds that one can process or generate in the mind in under a second) is incredibly fast, it follows immediately and logically without any need for an experiment that fMRI cannot have the temporal resolution needed for a rich understanding of the mind, simply based on knowing the TR (temporal sampling resolution) is so poor. And yet, I also find a lot of people in scientific brain research go oddly silent or seem to refuse to accept this argument unless some strange sort of published, quantificationist operationalization can be pointed to (hence my pre-emptive mentioning information transmission in neurons in under 100ms).

I'm not sure I'd call this scientism, exactly, I tend to see it as "selective quantificationism", i.e. that certain truths can only be proven as true if you introduce some kind of numerical measurement procedure and metrical abstraction. Like, no one demands a study with Scoville units to prove that e.g. a ghost pepper is at least an order of magnitude hotter than candied ginger, even though this is as blazingly obvious as the fact that the mind moves too fast for something that can barely capture images of the brain at a rate of two per second.

▲

throw4847285 6 days ago | parent [-]

I'm not a scientist, and I don't even have a very good statistical background, so correct me if I'm wrong; would it be far to say that the lack of skepticism about fMRI studies in the broader public is due to scientism? Because of naive reductionism and a gut understanding of what is "scientific", people are far more skeptical of a study that says, "we surveyed 100,000 people" vs. "we scanned the brains of 10 people." I've noticed a similar phenomenon with psych vs. evolutionary psych. People have an image in their head of what is scientific that has nothing to do with statistical significance and everything to do with vibes.

	▲	D-Machine 6 days ago \| parent [-]
		It is tempting to speculate on what might cause the credulousness of the broader public re: fMRI, but I think there is enough / too much going on here for me to really be able to say anything with much confidence. Scientism especially is hard to define. I think I broadly agree with you though that credulousness to (statistically and methodologically weak) scientific / technological claims mostly comes down to vibes and desires / needs, and not statistical significance, logical rigor, evidence, or etc. Where needs / desires are high, vibes will (often) win over rationality, and vice-versa. It is easier for people to be objective about science that doesn't really clearly matter in any obvious direction, or at all. fMRI is "the mind", and thus consciousness, and so unfortunately reduces rational evaluation in much the same way speculation about AI and "consciousness" and etc does. Shrug

▲

freehorse 5 days ago | parent | prev [-]

Yeah I agree mostly. Cognition happens in multiple timescales, as such I don't think that fmri's sampling rate is a problem if we understand which cognitive phenomena it can actually address and which not. But there is definitely a tendency to not understand such limits of our tools.

	▲	D-Machine 5 days ago \| parent [-]
		Precisely, if we restrict fMRI to investigating phenomena and theories of cognition and the mind that are plausibly measurable at the appropriate temporal resolution, it will potentially start yielding some fruit. It will also require fMRI researchers to think more carefully about their theories as well (e.g. noting the speed of the mind / amount and kinds of thinking involved in certain tasks, and being realistic about whether or not fMRI could actually capture something meaningful there). Too often there is no theory, and too many studies are just correlating patterns with some task without actually carefully thinking about the task and deconstructing the components, testing activations in those (e.g. ablation studies in AI research) and etc.

▲

physPop 5 days ago | parent | prev [-]

re: your last point that is not true. we can measure arbitrarially quickly (Nottingham group does some 3d EVI at ~100ms TRs). You can also reduce volumes and just look at single slices etc, a lot of the fundamental research did this (wash U / Minnesota / etc in the 90s). Its just not all that useful because the SNR tanks and the underlying neurovascular response is inherently low-pass. There is a much faster 'initial-dip' where the bold signal swings the other way and crosses zero (from localized accumulation of DeoxyHg before the inrush of OxyHg from the vascular response). Its a lot better correlated with LFP / spiking measures but just very hard to measure on non-research scanners...

	▲	D-Machine 5 days ago \| parent [-]
		Yes, I didn't mention this because you sacrifice so much spatial resolution and/or info doing this that it hardly matters, unless you believe in some very extreme and implausible forms of localization of function. (EDIT: I mean looking at a single slice seems to imply some commitment to localization assumptions; this isn't relevant for reducing spatial resolution.) For readers who don't know, we can measure at a higher temporal resolution better if we use some tricks, and also massively sacrifice spatial resolution ("reduce volumes") and/or how much of the brain is scanned (look at single slices), but the spatial resolution in most fMRI given e.g. a 0.5 TR (2 images per second) is usually already quite poor (generally already getting difficult to clearly even make out gyri and basic brain anatomy: see for example Figures 7 and on here, noting the TRs in the captions: https://www.frontiersin.org/journals/neuroscience/articles/1...). Still, it's a good point, and you're right of course newer and better scanners and techniques might improve things here on both fronts, but my understanding is that the magnetic field strengths needed to actually get the right combo of spatial and temporal resolution are, unfortunately, fatal, so we are really up against a physical/biological limit here. And as you said, it isn't that useful anyway, because the BOLD response is already so slow, and obviously something just emerging from the sum of a massive amount of far more rapid electrochemical signaling that the fMRI just can't measure anyway.