Beware "voodoo" brain science

A discussion among colleagues of a brain imaging study purporting to distinguish heterosexual from homosexual men prompted me to write on this topic. Perhaps even more controversial than categorizing sexual orientation, given the current legal climate, was the researchers' claim that their technique holds promise for identifying sexual deviants such as pedophiles and the those with sexual paraphilias.

Brain imaging is all the rage these days. The past decade has witnessed an explosion of interest in the fMRI, with literally thousands of studies, several new journals, and lavish federal funding and attention in the popular media. But some prominent neuroscientists express concerns about both the science and the ethics of fMRI research. Likening it to the old pseudo-science of phrenology, they caution that the public may be lured by vivid and colorful graphics into a misleading impression of scientific precision.

So, what is the fMRI?

Unlike the more established Magnetic Resonance Imaging (MRI) technique, which produces static images of the brain, the functional Magnetic Resonance Imaging (fMRI) provides images of the brain in action, or as it functions. The most widely used fMRI technique in cognitive neuroscience research is the BOLD (Blood Oxygen Level Dependent) method. This method is based on the premise that activation of specific brain regions affects blood flow and blood oxygenation, which can then be measured.

What does this have to do with forensics?

In the forensic arena, probably the most widely publicized research application of the fMRI is in the area of lie detection. fMRI data indicate that certain parts of people's brains -- specifically the anterior cingulate cortex and superior frontal gyrus -- are activated when they lie. But other studies show that the anterior cingulate activates during many other cognitive activities as well, indicating a lack of specificity that makes the technique inappropriate in the real world.

Other forensic applications include the hunt for the ever-elusive psychopath (the image at the left purports to show the brain of a psychopath in action). Some criminal defense attorneys also show fMRI images of their clients to jurors in an attempt to prove brain damage and thereby reduce their clients' legal culpability.

And, as I just said, researchers are starting to apply fMRI techniques to the study of human sexuality, including sexual orientation and sexual deviancy.

What are the problems?

Neuroscientist critics are issuing increasingly vocal alarm calls over both the underlying science and the practical applications of neuroimaging. Their central areas of concern include:

• The measurement techniques lack scientific precision
• Claims of scientific reliability and validity are overstated
• The fMRI overemphasizes brain localization when the brain functions more as a whole
• Applying group fMRI to individuals is improper at this stage of the science
• Marketing and forensic applications are ethically and philosophically problematic

Most recently, in a potentially landmark analysis, a group of psychologists based at the University of California at San Diego examined the analytical techniques used in 54 peer-reviewed fMRI brain-scanning studies published in prominent scientific journals. They concluded that the methods used in half the studies were so "seriously defective" that the findings "should not be believed."

Lead researcher Harold Pashler went so far as to call the statistical methods "voodoo" that should be especially shunned in the forensic arena:

"In the law, individual differences are the main focus," the Wall Street Journal quoted Pashler as saying. "And it often could come down to these voodoo statistics."

The article, "Puzzlingly High Correlations in fMRI Studies of Emotion, Personality,
and Social Cognition" (originally titled "Voodoo Correlations in Social Neuroscience") will be published in a forthcoming issue of the Association for Psychological Science's journal Perspectives in Psychological Science. Reports the abstract:

"Functional Magnetic Resonance Imaging studies of emotion, personality, and social cognition have drawn much attention in recent years, with high-profile studies frequently reporting extremely high (e.g., >.8) correlations between behavioral and self-report measures of personality or emotion and measures of brain activation. We show that these correlations often exceed what is statistically possible assuming the (evidently rather limited) reliability of both fMRI and personality/emotion measures. The implausibly high correlations are all the more puzzling because method sections rarely contain sufficient detail to ascertain how these correlations were obtained. We surveyed authors of 54 articles that reported findings of this kind to determine a few details on how these correlations were computed. More than half acknowledged using a strategy that computes separate correlations for individual voxels, and reports means of just the subset of voxels exceeding chosen thresholds. We show how this non-independent analysis grossly inflates correlations, while yielding reassuring-looking scattergrams. This analysis technique was used to obtain the vast majority of the implausibly high correlations in our survey sample. In addition, we argue that other analysis problems likely created entirely spurious correlations in some cases. We outline how the data from these studies could be reanalyzed with unbiased methods to provide the field with accurate estimates of the correlations in question. We urge authors to perform such reanalyses and to correct the scientific record."

Be sure to read this article (available HERE, as well as some of the related resources, below) before you get up on the witness stand and wax eloquent about the wonders of brain-scanning technology. Otherwise, on cross-examination you might be in for a nasty surprise.

Related resources:

• Law and Ethics of Brain Scanning, resources from a 2007 conference at Sandra Day O’Connor College of Law
• Law and Neuroscience Project, John D. and Catherine T. MacArthur Foundation
• "The Brain, Your Honor, Will Take the Witness Stand: Researchers Probe How the Mind Determines Crime and Punishment, but the Science Isn't Beyond a Reasonable Doubt," Wall Street Journal, Jan. 15, 2009
• "fMRI Beyond the Clinic: Will It Ever Be Ready for Prime Time?" by Richard Robinson, Public Library of Science Biology
• "Functional Magnetic Resonance Imaging (fMRI): Much Ado About What?" by Camilla Culler
• "Fact or Phrenology? The growing controversy over fMRI scans is forcing us to confront whether brain equals mind," by David Dobbs, Scientific American, March 2005
• "Can fMRI Really Tell If You're Lying?" by Gary Stix, Scientific American, August 2008
• "Neuroscience and the Law: If scientists can prove that the brain determines the mind, lawyers could convince juries that defendants may not be responsible for their crimes," By Michael S. Gazzaniga and Megan S. Steven, Scientific American, April 2005
• "The New Phrenology: The Limits of Localizing Cognitive Processes in the Brain,” by William R. Uttal

PS: My followup post on an attempt to get the "No Lie MRI" introduced in a Southern California parental termination case is HERE.