Debacles mar “Big Science” and fMRI research
ashington, 1989: “Now, Therefore, I, George Bush, President of the United States of America, do hereby proclaim the decade beginning January 1, 1990, as the Decade of the Brain.” Twenty-four years later: “Last year, I launched the BRAIN Initiative [a twelve year program] to help unlock the mysteries of the brain, to improve our treatment of conditions like Alzheimer’s and autism and to deepen our understanding of how we think, learn and remember.“ US-President Barack Obama.
Neuroscience is a wide field of brain and spine studies that evolved from the sciences of neuroanatomy and neurophysiology. Nowadays the discipline encompasses subdisciplines far outside the boundaries of the exact sciences and of medicine.
With the US-initiatives and those proposed and implemented elsewhere in the world, there came “Big Science” devoted to neuroscience research: a hundred million here, a hundred million there – or, in a European program, one billion over ten years. Altogether, many billions of euros or US-dollars were promised to subsidize projects heavily relying on fMRI research, among them the US-American Human Connectome Project and the European Human Brain Project. The ten-year European project started in 2013, but was already interrupted in 2015 for reasons described without overenthusiastic detail by the project leaders and by those responsible in the European Commission – but told fully in a paper in the Scientific American: “Two years in, a $1- billion-plus effort to simulate the human brain is in disarray. Was it poor management, or is something fundamentally wrong with Big Science?” . Those in the know in Europe remained silent.
In a report about the US Connectome project, one finds the following statement, hidden in a box outside the running text: “What is needed to get past the current impasse is a method that selectively separates global signal from global noise. Though no such method is yet available, we offer several observations about global fMRI fluctuations.” 
The history of MRI is a story of successes but also a story of empty promises.
The history of MRI is a story of successes but also a story of empty promises. Certain events have implications and consequences that only slowly unfold over the years.
BOLD imaging and its applications have developed into more than a disappointment – it might become an utter fiasco. Many results are based upon obscure metadata. I have described some reasons and low points in my last column about fMRI: “Functional charlatans” . I don't want to stray off here into the multifaceted picture of BOLD imaging and fMRI; on the other hand, I want to stress that there are numerous serious, genuine, and critical scientists in imaging neuroscience.
However, not only these scientists are upset, the educated and critical public is also getting annoyed, as Manfred Schneider describes their reactions to fMRI and Big Science in the science pages of the Swiss paper Neue Zürcher Zeitung:
“Numerous neuroscience institutes were founded, immense amounts of money were mobilized, the western societies got into a neuroscience frenzy. Thousands of subjects were placed in functional magnetic resonance equipment, where they had to endure movies, pornographic pictures, poems, while the researchers at their terminals observed the oxygen metabolism in their brain cells, converted the thrown out data into little pictures and tried to delude the world into believing that they are watching the brain thinking, feeling, acting. By now such studies have lapsed into a kind of oddball science. Meanwhile, no linguist can talk any more about synonyms without telling the confused zeitgeist that the words cake and pie are “synomynously” stored directly next to each other in the left temporal lobe .”
More and more researchers admit that acquisition and processing techniques of BOLD data lack the required meticulousness and thus the biased results and conclusions are scientifically irrelevant. Even the inventor of BOLD fMRI, Seiji Ogawa was among the harshest critics. Twenty-two years after his first description in 1990 , he published a 19-page review paper where he, in a roundabout way, discusses and disputes his technique .
Several thousand papers on fMRI appear every year, Kim and Ogawa mention 3,000 , PubMed's numbers stretch between 42,000 and nearly 170,000 since 1990, depending on the search terms one uses. Meanwhile it has become clear that many of these papers, apparently a majority, rest on shaky foundations. Some scientists read Seong-Gi Kim's and Seiji Ogawa's review as a farewell to BOLD imaging, but this conclusion seems too drastic and far-reaching. However, the publication alludes to the intricacy of the scientific background: “The BOLD effect in fMRI is very complex, and this is still an area of intense research.”
The authors also observe in their concluding remarks: “Dynamic properties and magnitudes of BOLD functional responses are dependent on many physiological parameters as well as baseline conditions. In patients with neurovascular disorders, the BOLD response could be sluggish, or even decreased relative to baseline. This should not be interpreted simply as a decrease in neural activity, because neurovascular coupling may be hampered … Resting-state fMRI studies are widely performed, but its physiological source needs to be systematically investigated.”
At the end, the critical reader's conclusion is: BOLD and fMRI should stay in the hands of genuine scientists. Clinical and psychological or commercial applications should be limited to trained and principled researchers. Today, the concepts of fMRI rely on a great many hypotheses, calculations, and simulations; however, practical proof to establish the validity of these models lags behind. Twenty-six years after Ogawa's original publication, everything is still “panta rhei – everything is in flux”: definitely no hope for a trip to Stockholm, but rather “back to the drawing board”.
“We found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%. These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results ."
Definitely, “Biostatistics for Radiologists” is not sufficient to perform fMRI and to apply statistical processing to the results. On the other hand, fMRI is not in the hands of radiologists – fortunately, in this case.
Functional MRI seemed one of the most promising research techniques for and beyond neuroimaging: the true study of brain organization. Now we fear the waste of hundred of millions euros of research grants and the shattered remains of thousands of scientific papers. Since nobody really feels responsible or in charge it will be difficult to minimize the repercussions of this debacle.
1. Theil S. Why the human brain project went wrong – and how to fix it. Scientific American 2015; 313. 1 October 2015.
2. Glasser MF, Smith SM, Marcus DS, et al. The Human Connectome Project’s neuroimaging approach. Nature Neuroscience 2016; 19: 1175-1187.
3. Rinck PA. Functional charlatans. Rinckside 2015; 26,4: 9-11.
4. Schneider M. Geld für Science-Fiction –- die Hirnforschungshysterie. Neue Zürcher Zeitung 24 May 2013; pg. 23.
5. Ogawa S, Lee TM, Kay AR, Tank DW. Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc Natl Acad Sci USA 1990; 87: 9868-9872.
6. Kim S-G, Ogawa S. Biophysical and physiological origins of blood oxygenation level-dependent fMRI signals. Journal of Cerebral Blood Flow & Metabolism 2012; 32: 1188-1206.
7. Segebarth C, Belle V, Delon C, Massarelli R, Decety J, Le Bas JF, Décorps M, Benabid AL. Functional MRI of the human brain: predominance of signals from extracerebral veins. Neuroreport. 1994, 21;5: 813-816.
8. Logothetis NK. What we can do and what we cannot do with fMRI. Nature. 2008; 453 (7197): 869-878.
9. Eklund A, Nichols TE, Knutsson H. Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. PNAS 2016; 113, 28: 7900-7905.