Flawed science? Two efforts launched to improve scientific validity of psychological test evidence in court

There’s this forensic psychologist, we’ll call him Dr. Harms, who is infamous for his unorthodox approach. He scampers around the country deploying a bizarre admixture of obscure, outdated and unpublished tests that no one else has ever heard of.

Oh, and the Psychopathy Checklist (PCL-R). Dr. Harms never omits that. To him, everyone is a chillingly dangerous psychopath. Even a 30-year-old whose last crime was at age 15.

What’s most bizarre about Dr. Harms’s esoteric method is that he gets away with it. Attorneys may try to challenge him in court, but their protests usually fall flat. Judges rule that any weaknesses in his method should go to the “weight” that jurors give Dr. Harm’s opinions, rather than the admissibility of his tests.

Psychological tests hold a magical allure as objective truth. They retain their luster even while forensic science techniques previously regarded as bulletproof are undergoing unprecedented scrutiny. Based in large part on our briefcases full of tests, courts have granted psychologists unprecedented influence over an ever-increasing array of thorny issues, from future dangerousness to parental fitness to refugee trauma. Behind the scenes, meanwhile, a lucrative test-production industry is gleefully rubbing its hands all the way to the bank.

In other forensic “science” niches such as bite-mark analysis and similar types of pattern matching that have contributed to wrongful convictions, appellate attorneys have had to wage grueling, decades-long efforts to reign in shoddy practice. (See Radley Balko's The Cadaver King and the Country Dentist for more on this.) But leaders in the field of forensic psychology are grabbing the bull by the horns and inviting us to do better, proposing novel ways for us to self-police.

New report slams "junk science” psychological assessments

In one of two significant developments, a group of researchers today released evidence of systematic problems with the state of psychological test admissibility in court. The researchers' comprehensive survey found that only about two-thirds of the tools used by clinicians in forensic settings were generally accepted in the field, while even fewer -- only about four in ten -- were favorably reviewed in authoritative sources such as the Mental Measurements Yearbook.

Despite this, psychological tests are rarely challenged when they are introduced in court, Tess M.S. Neal and her colleagues found. Even when they are, the challenges fail about two-thirds of the time. Worse yet, there is little relationship between a tool’s psychometric quality and the likelihood of it being challenged.

Slick ad for one of a myriad of new psych tests.

“Some of the weakest tools tend to get a pass from the courts,” write the authors of the newly issued report, "Psychological Assessments in Legal Contexts: Are Courts Keeping 'Junk Science' Out of the Courtroom?”

The report, currently in press in the journal Psychological Science in the Public Interest, proposes that standard batteries be developed for forensic use, based on the consensus of experts in the field as to which tests are the most reliable and valid for assessing a given psycholegal issue. It further cautions against forensic deployment of newly developed tests that are being marketed by for-profit corporations before adequate research or review by independent professionals.

"Life or death" call to halt prejudicial use of psychopathy test

In a parallel development in the field, 13 prominent forensic psychologists have issued a rare public rebuke of improper use of the controversial Psychopathy Checklist (PCL-R) in court. The group is calling for a halt to the use of the PCL-R in the sentencing phase of death-penalty cases as evidence that a convicted killer will be especially dangerous if sentenced to life in prison rather than death.

As I’ve reported previously in a series of posts (here and here, for example), scores on the PCL-R swing wildly in forensic settings based on which side hired the expert. In a phenomenon known as adversarial allegiance, prosecution-retained experts produce scores in the high-psychopathy range in about half of cases, as compared with less than one out of ten cases for defense experts.

Research does not support testimony being given by prosecution experts in capital trials that PCL-R scores can accurately predict serious violence in institutional settings such as prison, according to the newly formed Group of Concerned Forensic Mental Health Professionals. And once such a claim is made in court, its prejudicial impact on jurors is hard to overcome, potentially leading to a vote for execution.

The "Statement of Concerned Experts," whose authors include prominent professionals who helped to develop and test the PCL-R, is forthcoming from the respected journal Psychology, Public Policy, and Law.

Beware the all-powerful law of unintended consequences

This scrutiny of how psychological instruments are being used in forensic practice is much needed and long overdue. Perhaps eventually it may even trickle down to our friend Dr. Harms, although I have a feeling it won't be before his retirement.

But never underestimate the law of unintended consequences.

The research group that surveyed psychological test use in the courts developed a complex, seemingly objective method to sort tests according to whether they were generally accepted in the field and/or favorably reviewed by independent researchers and test reviewers.

Ironically enough, one of the tests that they categorized as meeting both criteria – general acceptance and favorable review – was the PCL-R, the same test being targeted by the other consortium for its improper deployment and prejudicial impact in court. (Perhaps not so coincidentally, that test is a favorite of the aforementioned Dr. Harms, who likes to score it high.)

The disconnect illustrates the fact that science doesn’t exist in a vacuum. Psychopathy is a value-laden construct that owes its popularity in large part to current cultural values, which favor the individual-pathology model of criminal conduct over notions of rehabilitation and desistance from crime.

It’s certainly understandable why reformers would suggest the development of “standard batteries … based on the best clinical tools available.” The problem comes in deciding what is “best.”

Who will be privileged to make those choices (which will inevitably reify the dominant orthodoxy and its implicit assumptions)?

What alternatives will those choices exclude? And at whose expense?

And will that truly result in fairer and more scientifically defensible practice in the courtroom?

It’s exciting that forensic psychology leaders are drawing attention to the dark underbelly of psychological test deployment in forensic practice. But despite our best efforts, I fear that equitable solutions may remain thorny and elusive.

Forensnips aplenty, forensnips galore

Everybody knows that the war is over
Everybody knows the good guys lost
Everybody knows the fight was fixed
The poor stay poor, the rich get rich
That's how it goes, Everybody knows

I can't seem to get Leonard Cohen’s haunting Everybody Knows out of my mind.

Perhaps it's because I was just down in Alabama, the belly of the beast, working on a tragic case. With the highest per capita rate of executions in the United States, the Heart of Dixie State kills people for crimes that other nations punish with probation. No exaggeration. It was jarring to drive around  Montomery and see the close proximity of historic mansions to abandoned homes and decaying housing projects. The juxtaposition is fitting, as Montgomery claims the dual distinctions of being the "cradle of the Confederacy" and the "birthplace of the Civil Rights Movement."  

Montgomery, Alabama (c) Karen Franklin 2013

Or maybe it's a flashback to Elysium, in which the one percenters have left Earth’s teeming masses to rot away while they luxuriate on an idyllic orbiting satellite. The scene in the parole office, with a robot parole agent delivering a quick risk assessment and then pushing meds, is worth the price of admission, although the film is marred by interminable hand-to-hand combat scenes and a ridiculous Hollywood ending.

David Miranda, held hostage
by British security forces

Or, it could be because I’m still riled up over the British government's abuse of David Miranda. He is the Brazilian partner of journalist Glenn Greenwald (think Edward Snowden). In what can only be called an outrageous effort to intimidate journalists, the Brits detained Miranda at Heathrow Airport for nine solid hours -- the maximum allowed under the British Terrorism Act -- before finally releasing him sans his laptop, cell phone and camera. Under the Terrorism Act, he was not entitled to counsel, nor to decline to cooperate. I sure hope it backfires and incenses journalists; it certainly fired up USA Today columnist Rem Rieder (whose column I highly recommend).

* * * * *

I feel bad about the dearth of posts recently. It's been a hectic period. I'll try to make up for my lapse by packing this post with lots of links to forensic psychology and criminology news and views from the past few weeks:

Evidence-based justice: Corrupted memory

Nature magazine's profile of Elizabeth Loftus and her decades-long crusade to expose flaws in eyewitness testimony is worth a gander.

Psychopathic criminals have empathy switch

New research published in the journal Brain indicates that psychopaths do not lack empathy, as is often claimed. Rather, they can switch it on and off at will. The study, out of the Netherlands Institute for Neuroscience, is freely available online. BBC also has coverage.  

The demographics of sexting

Sexting is becoming increasingly commonplace. But practices and meanings differ by gender, relationship and sexual identity, according to a new article, also available online, in the journal Cyberpsychology, Behavior, and Social Networking.

Brainwashed video discussion

New York Times columnist David Brooks just interviewed psychiatrist Sally Satel and psychologist Scott Lilenfield about their new book, Brainwashed, which is getting quite a bit of media buzz. The book is a workmanlike, if a bit superficial, exploration of the allure of "mindless neuroscience." If you’ve got 65 minutes, I recommend watching the video discussion.

Prison news: Hunger strike, juveniles, the elderly, women

On the prison front, a lot has been going on. California prisoners are into Day 50 or so of their hunger strike over solitary housing (a condition that the Department of Corrections denies, despite many men being kept in segregation units for years and even decades) and other cruel conditions. With prisoners' health deteriorating, a court order has been issued allowing force feeding if necessary to forestall deaths. Mainstream media reporting has been minimal, but at least Al Jazeera's got you covered.  

Even more local to me, a lawsuit has been filed over solitary confinement of juveniles in Contra Costa County. The lawsuit, filed in U.S. District Court, accuses county officials of flouting state laws mandating that juvenile detention facilities be supportive environments designed for rehabilitation.

Meanwhile, NBC news is sounding an alarm over the increasing number of elderly people in U.S. prisons. NBC sounds mostly worried about the cost to taxpayers of prisons teeming with upwards of 400,000 elderly prisoners by the year 2030. Read ithttp://usnews.nbcnews.com/_news/2013/06/29/.UeV62HppQL8.twitter, and weep. 

Piper Kerman, author of the memoir Orange Is the New Black that's become a trendy Netflix series, is also sounding an alarm. In a New York Times op-ed, she writes about a federal plan to ease overcrowding in men's prisons by shipping about 1,000 women from Connecticut down to Alabama and points beyond, where they will be even more estranged from their families. As Kerman notes: "For many families these new locations might as well be the moon." I recommend her thoughtful essay on alternatives for low-risk women prisoners. 

In a more promising development, the U.S. Justice Department has announced efforts to curtail the stiff drug sentences that have caused much of this overcrowding in the first place. The U.S. prison system is so bloated, so costly, and so irrational, that even conservatives are calling for reform. Better late than never, I suppose.

By the way, Florida has executed John Errol Ferguson, the prisoner whose controversial case I blogged about earlier this year, whose competency was contested in part because of his insistence that he was the "Prince of God." The American Bar Association had filed an amicus brief asking the U.S. Supreme Court to clarify the standard for competency for execution being applied in the case. 

Sex offender news

In yet another in a series of registry-facilitated vigilante attacks, a South Carolina man has been arrested for killing a sex offender and his wife in the mistaken belief that the man was a child molester. At the same time, there are signs that overzealous laws that contribute to such stigmatization are being scrutinized more closely. For example, a federal judge has struck down a Colorado city's ordinance restricting where registered sex offenders can live, ruling that it conflicts with a state law requiring parolees to be reintegrated into society. An appellate panel in North Carolina has also struck down a law that banned registered sex offenders from using social media sites. The state Court of Appeals agreed with the challenger that the law violated his Constitutional rights to free speech and freedom of association. 

Dispute over expert witness credentials

Finally, there's a big brouhaha in South Dakota over the credentials of a psychologist who frequently testifies as an expert witness in child custody cases. The credentials of the widely respected psychologist, Thomas Price, became an issue during a child custody dispute. It was ascertained that he had earned his PhD in behavioral medicine from an online degree mill called Greenwich University on Norfolk Island, Australia, that was subsequently shuttered by the Australian government. According to an expert on diploma mills quoted by the Sioux Falls Argus Leader, degree mills often adopt the names of respected English universities. Price's resumé says he earned a Ph.D. in behavioral medicine from Greenwich University, without noting the Norfolk Island location. "Typically," notes the article, "people don’t get caught using an unaccredited degree until they assume a high-profile position ... or they do something that causes another person to research their backgrounds…. If you stay under the radar, you can get by."

Science blogger

Finally (this time I really mean it), for those of you who are into offbeat science, I've just added a new blog, Mike the Mad Biologist, to my blog roll (which can be found a little ways down the right column of my blog site). Mike is prolific and wide-ranging in his news links, with a creative spin. 

Hat tips to Jane, Terry, Kirk and others

'Digital lynch mob' assaults expert witness in televised murder trial

Imagine you are testifying in a high-profile murder case being live-streamed over the Internet. Suddenly, an angry mob swarms all over you. More than 10,000 people sign an online petition urging a boycott of your lecture contracts. Your book gets a thousand negative hits on Amazon. You are stalked, and a photo of you dining with the trial attorney is posted on Facebook, implying unethical conduct. You even get death threats.

That is the social media-coordinated avalanche that hit domestic violence expert Alyce LaViolette, testifying for the defense in the capital murder trial of Jodi Arias. The unrelenting cyber assaults so rattled LaViolette that she suffered an anxiety attack that landed her in the emergency room.

But the ER visit may only encourage the cyber-stalkers, who revel online over her discomfiture and obvious emotional deterioration over the course of seven grueling days of court testimony.

This type of Internet mobbing, in which cyber-posses enforce social norms through public shaming, is becoming more and more commonplace. One of the most widely known examples of such Internet vigilanteism was the 2005 case of "Dog Poop Girl," a South Korean woman who gained infamy after she refused to clean up after her dog on a Seoul subway; the harassment eventually escalated to the point that she was forced to quit her university job. 

But what was LaViolette's crime?

The domestic violence counselor had the audacity to opine that Jodi Arias was a victim of domestic violence -- that she was dominated and abused (physically, emotionally and sexually) by the man she eventually killed. Such an opinion bolsters Arias's claim that she killed her ex-boyfriend in self defense.

Murder tragedies as entertainment

Unfortunately for LaViolette, her analysis runs counter to the dominant narrative in a gendered morality play produced by media conglomerate Turner Broadcasting and distributed through its cable channels HLN, CNN and In Session. In this good-versus-evil melodrama, Arias is a psychopathic female who killed a morally righteous man in a fit of jealous rage. Period. End of story. Airbrushed out are all the nuances, the shades of grey inevitably present in any such violent tragedy. 

The burgeoning infotainment industry has perfected a profit-making formula of sensationalized true-crime "reporting" that plays on viewers' emotions, whipping audiences into a frenzy of self-righteous indignation in which they clamor for guilty verdicts -- very often against female transgressors. Nancy Grace's shrill ranting over the Casey Anthony murder acquittal garnered HLN a record of almost three million viewers. More recently, HLN went after another woman, Elizabeth Johnson, suspected in the mysterious disappearance of her baby. 

The Arias case seems Heaven-sent for this voyeuristic style of entertainment, in which vulturous pundits mete out tantalizing morsels of crime "facts" to their addicted audience. Travis Alexander provides titillation from the grave via thousands of graphic emails, instant messages, texts and phone chats in which he degrades his paramour as a "whore," "slut," "corrupted carcass" and "three-hole wonder" whom he can sexually violate at will. For her part, Arias is a demonstrable liar. When her ex-boyfriend was found with a gunshot wound to the head, a slit throat, and more than two dozen stab wounds, she initially claimed innocence. After police demolished her alibi defense, she then claimed that two intruders broke into the home and killed Alexander, before finally admitting to the killing but claiming self defense.

Cast in the starring role of swashbuckling hero in this sordid drama is prosecutor Juan Martinez, a dapper man with a quick mind and an acerbic style, whose meteoric rise from the son of Mexican immigrants to a top government attorney is the stuff of American legend. Women line up outside the Maricopa County, Arizona courthouse, swooning at the sight of him as they jockey for photographs and autographs.

"This is murder trial as entertainment," Josh Mankiewicz, a correspondent for NBC's Dateline program (which ran two segments on the case), told reporter Michael Kiefer of the Arizona Republic. "This is not a trial like O.J. (Simpson's) that sheds new light on society. This is not about race or money. It's a perfect tabloid storm. It is occurring in the absence of any other tabloid storm."

Nancy Grace, "Dr. Drew" and the other pundits capitalizing on such trials foster a false sense of intimacy by calling everyone by first names. They encourage vicarious audience participation on Facebook, Twitter, online polls and other social media. But this is no value-neutral production. This is an archetypal trope that requires a guilty verdict; as one insightful media critic noted, acquittals do not produce the desired catharsis.

Public shaming run amok

In such an emotionally charged climate, anyone affiliated with the defense automatically becomes a villain. However, it is interesting to observe the disparate treatment of LaViolette as compared with a male expert witness, psychologist Richard Samuels. The prosecutor aggressively attacked them both. Playing not only to the jurors but to his sizeable out-of-court fan base, Martinez paced back and forth like a tiger smelling blood, demanding of his cornered prey that they give only "yes or no" answers to his myriad questions. Under his withering cross-examination, both witnesses came across as defensive and evasive. Both were vulnerable due to their confirmatory biases -- a failure to seek out evidence that might disconfirm their case theories. But, objectively, Samuels would seem to invite at least as much criticism as LaViolette, due to his bumbling style, his test scoring errors, and his questionable case formulation (he diagnosed posttraumatic stress disorder using a rating scale on which Arias endorsed a fictitious trauma, of witnessing Alexander's murder at the hands of imaginary intruders).

However, the public's palpable fury against LaViolette far outstrips that targeting Samuels. Consistent with the Turner Network's gendered narrative of criminal villainy, the cyber-posse is fueled by a potent combination of misogyny and homophobia: The expert witness in their crosshairs is "emasculating," "a bull dyke," "a man-hater," "fat," "buck-teethed," "a bitch."

The Internet fosters this culture of hate. Its cloak of anonymity is disinhibitory, emboldening people to spew bile with impunity. In The Cult of the Amateur, Andrew Keen warns that the deluge of anonymous online content is altering public debate, manipulating opinion, blurring the boundaries between experts and the uninformed and weakening the vitality of professional media -- newspapers, magazines, music and movies.

The proliferation of bottom-feeders on Twitter and YouTube is one thing. But it is quite another thing when cyber-bullying seeps into the courtroom, intimidating witnesses and threatening the presumption of innocence.

Can inundated jurors remain unbiased?

Legal experts worry that a virtual deluge of unreliable and biased information -- readily available at the click of the mouse or a TV remote -- is undermining jurors' neutrality. In their off hours, curious jurors in the Arias case can tune in not only to the cable TV and social media debacle, but can watch the defendant's entire videotaped police interrogation -- including excised portions -- as well as a police interview with Arias's parents, in which they speak of her mental problems. Pro- and anti-Arias websites have sprung up. And it's not just outsiders who are furiously Tweeting, texting and blogging about the case.  Witnesses are watching the trial from home and texting the prosecutor with suggestions for cross-examination. Jodi Arias herself is tweeting from the jail, through a friend. ("HLN is an acronym for Haters Love Negativity," she tweeted.)

It would be naive to suppose that the Arias jury is immune to the inflammatory rhetoric swirling around the Internet. Some of the more sarcastic questions that jurors submitted for the expert witnesses sounded scripted by Nancy Grace. For example, one juror asked psychologist Samuels whether a bad haircut could induce posttraumatic stress disorder (PTSD), Samuels's diagnosis for Arias.

Yet trial judge Sherry Stevens -- who allowed cameras into the courtroom in the first place -- is now relying on the honor system rather than regaining control by sequestering the jury.  Complained defense attorney Kirk Nurmi: "The court asks the question of the jurors every morning, 'Have you seen anything on the media?' No one raises their hand... It is a fairy tale to assume that this jury is not hearing any of this. It is all over the news."

Kiefer, the Arizona Republic reporter who broke the story of witness LaViolette's cyber-bullying, gave examples of juror social-networking misconduct in other cases: A Michigan juror who posted a Facebook preview of her verdict ("Gonna be fun to tell the defendant they're GUILTY"); a juror in Britain who polled her social-media "friends" as to whether she should find a defendant guilty.

With more and more successful appeals of verdicts due to such Internet or social-media interference, according to a Reuters Legal survey, an appeal of any guilty verdict in the four-month Arias trial is a virtual certainty.

But any appeal will not mend the reputations of the expert witnesses called by the defense. As a retired Maricopa County Superior Court judge told Michael Kiefer, the Arizona Republic reporter, "it's the electronic version of a lynch mob."

Sree Sreenivasan, a journalism professor at Columbia University, told Kiefer he had never seen anything like the attack on LaViolette, but that it likely will become "standard operating procedure in prominent cases" -- witness intimidation taken to its logical extreme in a public culture of shaming and vilification.

If so, experts may think long and hard before about accepting referrals in high-profile cases. That, in turn, could have a chilling effect on defendants' rights to a fair trial.

Michael Kiefer's insightful Arizona Republic reports on the social media debacle are HERE, HERE and HERE. A full collection of the live-streamed trial videos is located HERE.

Remarkable experiment proves pull of adversarial allegiance

 Psychologists' scoring of forensic tools depends on which side they believe has hired them

A brilliant experiment has proven that adversarial pressures skew forensic psychologists' scoring of supposedly objective risk assessment tests, and that this "adversarial allegiance" is not due to selection bias, or preexisting differences among evaluators.

The researchers duped about 100 experienced forensic psychologists into believing they were part of a large-scale forensic case consultation at the behest of either a public defender service or a specialized prosecution unit. After two days of formal training by recognized experts on two widely used forensic instruments -- the Psychopathy Checklist-R (PCL-R) and the Static-99R -- the psychologists were paid $400 to spend a third day reviewing cases and scoring subjects. The National Science Foundation picked up the $40,000 tab.

Unbeknownst to them, the psychologists were all looking at the same set of four cases. But they were "primed" to consider the case from either a defense or prosecution point of view by a research confederate, an actual attorney who pretended to work on a Sexually Violent Predator (SVP) unit. In his defense attorney guise, the confederate made mildly partisan but realistic statements such as "We try to help the court understand that ... not every sex offender really poses a high risk of reoffending." In his prosecutor role, he said, "We try to help the court understand that the offenders we bring to trial are a select group [who] are more likely than other sex offenders to reoffend." In both conditions, he hinted at future work opportunities if the consultation went well. 

The deception was so cunning that only four astute participants smelled a rat; their data were discarded.

As expected, the adversarial allegiance effect was stronger for the PCL-R, which is more subjectively scored. (Evaluators must decide, for example, whether a subject is "glib" or "superficially charming.") Scoring differences on the Static-99R only reached statistical significance in one out of the four cases.

The groundbreaking research, to be published in the journal Psychological Science, echoes previous findings by the same group regarding partisan bias in actual court cases. But by conducting a true experiment in which participants were randomly assigned to either a defense or prosecution condition, the researchers could rule out selection bias as a cause. In other words, the adversarial allegiance bias cannot be solely due to attorneys shopping around for simpatico experts, as the experimental participants were randomly assigned and had no group differences in their attitudes about civil commitment laws for sex offenders.

Sexually Violent Predator cases are an excellent arena for studying adversarial allegiance, because the typical case boils down to a "battle of the experts." Often, the only witnesses are psychologists, all of whom have reviewed essentially the same material but have differing interpretations about mental disorder and risk. In actual cases, the researchers note, the adversarial pressures are far higher than in this experiment:

"This evidence of allegiance was particularly striking because our experimental manipulation was less powerful than experts are likely to encounter in most real cases. For example, our participating experts spent only 15 minutes with the retaining attorney, whereas experts in the field may have extensive contact with retaining attorneys over weeks or months. Our experts formed opinions based on files only, which were identical across opposing experts. But experts in the field may elicit different information by seeking different collateral sources or interviewing offenders in different ways. Therefore, the pull toward allegiance in this study was relatively weak compared to the pull typical of most cases in the field. So the large group differences provide compelling evidence for adversarial allegiance."

This is just the latest in a series of stunning findings by this team of psychologists led by Daniel Murrie of the University of Virginia and Marcus Boccaccini of Sam Houston University on an allegiance bias among psychologists. The tendency of experts to skew data to fit the side who retains them should come as no big surprise. After all, it is consistent with 2009 findings by the National Academies of Science calling into question the reliability of all types of forensic science evidence, including supposedly more objective techniques such as DNA typing and fingerprint analysis.

Although the group's findings have heretofore been published only in academic journals and have found a limited audience outside of the profession, this might change. A Huffington Post blogger, Wray Herbert, has published a piece on the current findings, which he called "disturbing." And I predict more public interest if and when mainstream journalists and science writers learn of this extraordinary line of research.

In the latest study, Murrie and Boccaccini conducted follow-up analyses to determine how often matched pairs of experts differed in the expected direction. On the three cases in which clear allegiance effects showed up in PCL-R scoring, more than one-fourth of score pairings had differences of more than six points in the expected direction. Six points equates to about two standard errors of measurement (SEM's), which should  happen by chance in only 2 percent of cases. A similar, albeit milder, effect was found with the Static-99R.

Adversarial allegiance effects might be even stronger in less structured assessment contexts, the researchers warn. For example, clinical diagnoses and assessments of emotional injuries involve even more subjective judgment than scoring of the Static-99 or PCL-R.

But ... WHICH psychologists?!

For me, this study raised a tantalizing question: Since only some of the psychologists succumbed to the allegiance effect, what distinguished those who were swayed by the partisan pressures from those who were not?

The short answer is, "Who knows?"

The researchers told me that they ran all kinds of post-hoc analyses in an effort to answer this question, and could not find a smoking gun. As in a previous research project that I blogged about, they did find evidence for individual differences in scoring of the PCL-R, with some evaluators assigning higher scores than others across all cases. However, they found nothing about individual evaluators that would explain susceptibility to adversarial allegiance. Likewise, the allegiance effect could not be attributed to a handful of grossly biased experts in the mix.

In fact, although score differences tended to go in the expected direction -- with prosecution experts giving higher scores than defense experts on both instruments -- there was a lot of variation even among the experts on the same side, and plenty of overlap between experts on opposing sides.

So, on average prosecution experts scored the PCL-R about three points higher than did the defense experts. But the scores given by experts on any given case ranged widely even within the same group. For example, in one case, prosecution experts gave PCL-R scores ranging from about 12 to 35 (out of a total of 40 possible points), with a similarly wide range among defense experts, from about 17 to 34 points. There was quite a bit of variability on scoring of the Static-99R, too; on one of the four cases, scores ranged all the way from a low of two to a high of ten (the maximum score being 12).

When the researchers debriefed the participants themselves, they didn't have a clue as to what caused the effect. That's likely because bias is mostly unconscious, and people tend to recognize it in others but not in themselves. So, when asked about factors that make psychologists vulnerable to allegiance effects, the participants endorsed things that applied to others and not to them: Those who worked at state facilities thought private practitioners were more vulnerable; experienced evaluators thought that inexperience was the culprit. (It wasn't.)

I tend to think that greater training in how to avoid falling prey to cognitive biases (see my previous post exploring this) could make a difference. But this may be wrong; the experiment to test my hypothesis has not been run. 

The study is: "Are forensic experts biased by the side that retained them?" by Daniel C. Murrie, Marcus T. Boccaccini, Lucy A. Guarnera and Katrina Rufino, forthcoming from Psychological Science. Contact the first author (HERE) if you would like to be put on the list to receive a copy of the article as soon as it becomes available.

Click on these links for lists of my numerous prior blog posts on the PCL-R, adversarial allegiance, and other creative research by Murrie, Boccaccini and their prolific team. Among my all-time favorite experiments from this research team is: "Psychopathy: A Rorschach test for pychologists?"

Documentary targets family courts and custody evaluators

Stuck in the middle of nowhere on a case, I happened to catch the new documentary No Way Out But One, depicting injustices against abused women and children in U.S. family courts. Of potential interest to blog readers, the film critiques the role of child custody evaluators as usurping the authority of fact finders by substituting their own judgments for the facts.

No Way Out highlights the internationally known case of Holly Collins, who fled with her son Zachary and daughter Jennifer in 1994 after her husband was granted sole custody by a court in Minnesota. According to the film, the judge ignored evidence of domestic violence and child abuse, including a skull fracture to the boy. After a circuitous flight through Canada and Guatemala, Collins eventually won asylum in the Netherlands. By the time the FBI caught up with the family, the children were adults. In the film, they convincingly describe chronic abuse at the hands of their father. Holly's dynamic daughter, Jennifer, the inspiration for the film, is executive director of Courageous Kids, which empowers children to go public about family court abuse (her blog is HERE).

The Collins children, grown up

The Collins's long-running custody battle featured allegations of Parental Alienation Syndrome, a controversial syndrome in which one parent (most often the mother) is accused of alienating the children from the other parent. Collins was also labeled with another highly contentious diagnosis, Munchausen by Proxy, after she sought medical treatment for her children, whom she says were being injured by their father’s abuse and neglect.

Due in part to Collins's supposed attempts to alienate the children, the father was granted full custody in 1993, and Collins was initially denied even phone or mail contact. Eventually, she was granted supervised visitation, but neither she nor her children were allowed to talk about the father’s abuse. In the film, Collins describes how she and the children secretly exchanged notes by placing them in the refrigerator; in the notes, the children begged for help and she finally promised to rescue them.

Collins became the first American ever granted asylum by the Netherlands. She ultimately married a Dutch man and had four more children. After the FBI located her, she returned to the United States in an effort to vindicate herself. Ultimately, the kidnapping charge was dismissed; she pled guilty to one count of contempt of court in exchange for a sentence of 40 hours of community service.

According to the film, Collins is just one of thousands of mothers forced to go on the run in order to protect their children from abusive fathers who have been granted custody of their children.

Jennifer (L) and Holly Collins (R) with filmmakers Nolan and Waller

Ironically, the film's debut on the Documentary Channel coincides with the publication of a similar story by another woman who is also named Collins. Frances Collins's book, Seashell Prisoners, chronicles her flight from Texas to the Honduras to protect her 3-year-old granddaughter. Her eight-year odyssey ultimately ended in arrest and incarceration.

The film is stoking up antipathy between the battered women's and father's rights camps, with the latter expending significant effort in to debunk the claims of Collins and her children that they were subjected to family violence.

Award-winning filmmaker Garland Waller told a Huffington Post columnist that she chose the Holly Collins case "because I believed her story would break through the barricade set up by the mainstream press." The film expands on last year’s award-winning short, Small Justice, produced on a shoestring by Waller, a communications professor at Boston University, and her husband Barry Nolan, a TV writer and reporter.

In the Huffington Post interview, Waller went on to say that what most surprised her in her involvement with this project was the dumbfounded reaction of members of the general public:

"They just can't believe that … family courts would give custody -- time and time again -- to abusers. But I suppose I really shouldn't be surprised. In both the tragedy of the Catholic Church sex abuse scandal and the Jerry Sandusky thing, ‘good’ people turned a blind eye to the abuse of children. It's the same thing in family courts. It is just heart-breaking that so often when terrified children summon the courage to speak up and tell what is happening to them, even though the abuser has warned them of the terrible consequences if they ever talk... even though we teach children to speak up and to tell the truth...when they speak up against this one awful thing, we just don't listen."

I don’t see any more upcoming airings on the DocumentaryChannel, but the DVD will be going on sale soon, from Passion River Films.  

For people who are trying to stay positive and collaborative while going through a stressful divorce, a Florida law firm has put together a set of helpful tips from top relationship experts: "Coping with Divorce." 

Jury Expert: Timely focus on false confessions

"Why on earth would anyone, anywhere, ever confess to a serious crime they did not commit? Especially something like murder? Seriously? Mock jurors find this hard to believe." In the latest issue of the Jury Expert, two trial consultants present the research on why people falsely confess and the cascade of errors that follow a false confession.

The release of the documentary The Central Park Five makes Douglas Keene and Rita Handrich's overview, "Only the Guilty Would Confess to Crimes": Understanding the Mystery of False Confessions, especially timely. The consultants provide a concise summary of current research findings and references, making it an excellent resource for criminal attorneys, investigators, judges, law enforcement officials, students of law and public policy and, of course, forensic professionals. It is followed by commentaries from widely renowned false confession expert Saul Kassin and three others, including your blogger.

My essay, "Disputed confessions: The many hats of the expert witness," may be of special interest to this blog's readers, as it details the role of the expert witness in cases involving disputed confessions. I discuss the distinctions between "pure" academic research and clinical assessment, and the role of the forensic expert in evaluating both psychological vulnerabilities that might contribute to an unreliable confession and the separate issue of a suspect's comprehension of the mandatory Miranda rights waiver.

In an accompanying feature in the November/December issue, jury consultant Diane Wiley of the National Jury Project provides a supplemental jury questionnaire covering the issues attorneys need to address in a false confessions case.

And there's even more on the confessions topic in the new issue, hot off the press today: Rita Handrich reviews the 2010 volume, Police Interrogations and False Confessions: Current Research, Practice, and Policy Recommendations, edited by well-known scholars Daniel Lassiter and Christian Meissner.

The main feature (with accompanying essays) is HERE; my essay is HERE, and the entire current issue and back issues of the always-excellent Jury Expert can be accessed HERE. For you tweeters, the Jury Expert's Twitter feed is HERE.

Another one bites the dust: Hollow SVP prosecution no match for jurors' common sense

15 minutes.

After a five-week trial, that's how long it took a jury in a rural Northern California county to decide that an openly gay man who had served two years in prison for a forcible oral copulation of an acquaintance back in 2003 did not merit civil commitment as a sexually violent predator.

The prosecution's case featured a lone government psychologist whose opinion rested on a hollow combination of homophobia, bogus psychiatric diagnoses and trumped-up risk estimates. The psychologist cited archaic (and discredited) Freudian theory to claim that the ex-offender's crime at age 23 was evidence of an "oral incorporation" fixation caused by a domineering mother and an absent biological father. As a legal basis for civil commitment, he cited the bogus disorder of "paraphilia not otherwise specified-nonconsent,” and he used the Static-99R actuarial tool to present a highly inflated estimate of risk.

Testifying for the defense were four psychologists, including two retained by the defense, a government evaluator who had changed her mind (or "flipped," in the current parlance) and the man's treating psychologist at Coalinga State Hospital, who testified in no uncertain terms that "Mr. Smith," as I will call him, is neither mentally disordered nor likely to reoffend.

The defense team had barely left the courthouse when the court clerk summoned them back, saying the jury had reached a verdict. Their astonishingly fast decision hints that the jurors agreed that this case was an egregious example of overzealous prosecution and a waste of their valuable time.

Prior to being screened for possible civil commitment, Mr. Smith had been on parole in the community for 14 months without getting into any trouble whatsoever. Indeed, he was busy doing good works. His sexually violent predator screening stemmed from an entirely accidental parole violation connected with his charity work for a local gay rights organization. He had a special parole condition forbidding any contact with children. When a fellow member of the executive board brought his child to an awards ceremony, Mr. Smith was exposed to "incidental contact as one might have while shopping at a market," in the words of the parole hearing officer. Unfortunately for Mr. Smith, this was just one month after California voters enacted Jessica's Law, which allows for civil commitment of sex offenders who have only one qualifying victim rather than the previous minimum of two.

The prosecutor's strategy, as is typical in weak cases, was to hurl as many prejudicial, pseudoscientific labels as possible in Mr. Smith's direction, and hope a few might stick and scare jurors into voting for civil commitment: Psychopath, antisocial, homosexual, paraphilic, high risk, etc.

While licensed as a psychologist, the government's expert had not done what clinical psychologists are trained to do: Psychological testing, individualized case formulation, etc. Rather, as he boldly admitted on the witness stand, he relied on an assistant to cull through Mr. Smith's hospital records and pull out negative behavioral reports for him to review. Wow! Can you spell B-I-A-S?

In my testimony, which stretched over the course of three days, I stressed that Mr. Smith was neither sexually deviant nor likely to reoffend. His risk of sexual reoffense, I testified, was no greater than that of any other garden-variety sex offender. (The base rate of sexual recidivism among convicted sex offenders in California -- similar to the rest of the United States -- hovers around 6 percent or less.) I explained how growing up gay in a homophobic family and community causes sexual identity confusion that can lead to sexual acting out and other delinquent behavior in adolescence and early adulthood, and how Mr. Smith had changed as he matured and accepted his sexuality. I further debunked the accuracy of the Static-99R "actuarial" risk estimates assigned in this case, and the pretextually applied diagnoses of "paraphilia not otherwise specified-nonconsent" (which I've blogged about repeatedly) and antisocial personality disorder, a red herring that was invoked despite Mr. Smith's exceptionally good conduct in the community and while in prison.

Stacking the deck

The prosecutor tried to stack the deck by striking from the jury all gay people or those who admitted having relatives or close friends who are gay; he also challenged those with advanced educational degrees. I guess he thought it would be easier to pull the wool over the eyes of an uneducated jury. It just goes to show that times have changed: Even in a rural county, antigay discrimination is no longer considered acceptable, and jurors don't need PhD's to recognize bias and pseudoscience when they hear it.  

The verdict was likely a bitter-sweet moment for Mr. Smith, who had spent more than four years incarcerated at Coalinga awaiting trial. Luckily, he has close friends to stay with while getting on his feet.

This is my third SVP case in a row that evaporated when finally exposed to the light of day. Like Mr. Smith's case, one of the other two also featured prominent antigay bias; the other targeted an immigrant. In neither case were the men either pedophiles or rapists.

I suppose I should feel pleased to see such gross miscarriages of justice thwarted. Instead, I find myself horrified by the unfettered power wielded by rogue psychologists, assigned to a case by luck of the draw. Whereas many government evaluators reserve "positive" findings for the rare sex offenders who are truly deviant and at high risk to reoffend, others are just hacks who are raking in obscene amounts of public funds while making little effort to truly understand these men, their motivations, their circumstances, or their pathways to desistance.

Especially frightening is the unconscious bias that creeps into SVP prosecutions. The constructs of "mental disorder" and "risk for reoffense" are malleable, lending themselves to use as pretextual weapons of prejudice wielded against gay men, racial minorities (especially African American men) and immigrants.

Clearly, people shouldn't get away with sexual misconduct. But none of these men had. All had pleaded guilty and served their time, only to be ambushed at the end of their prison terms with misguided efforts to indefinitely detain them based on purported future risk.

As it turned out, each case was about as solid as a house of cards. It didn't take gale-force winds like Hurricane Sandy's to flatten them.

Evaluators flipping like pancakes

The "flipping" of government evaluators illustrated this weak foundation. In two of the three cases, after reading the more thorough and individualized reports of the defense-retained experts, government psychologists abruptly changed their minds and decided that their previously proffered diagnoses of "paraphilia not otherwise-nonconsent" were invalid.

On the one hand, I applaud the openness and ethical backbone such a change of heart signals. But these "flips" also demonstrate the whimsical, nonscientific nature of the commitment process. The longer I work in these trenches, the more I realize that the random assignment of evaluators and attorneys (on both sides) exerts as much influence on the outcome as does the true level of future risk to the community that an ex-offender poses.

Indeed, the real reason Mr. Smith -- clearly not a sexual predator to anyone with a whit of commons sense -- was taken to trial, at a total cost to the citizenry of hundreds of thousands of dollars, was not because of his high risk, but because of a rigid prosecutor who was blind to the writing on the wall.

In contrast, the government dismissed the other two cases (one in the Midwest and one in the South) on the eve of trial. One case involved a gay man who had a brief sexual interlude with a teenage male relative; the other involved an immigrant who had gone on two dates with an underage teen girl he met on an online dating site (his misconduct never went beyond petting). Both had served substantial prison terms. But, again, garden-variety sex offenders, not the depraved, sex-crazed monsters likely envisioned by jurors when they are told they will be deciding a "sexually violent predator" case.

Bottom line: Should a random clinical psychologist, earning hundreds of thousands of dollars a year churning out boilerplate pseudoscientific garbage, be allowed to decide the fates of others?

At least in this one case, 12 discerning and conscientious jurors answered that question with a resounding "NO."


ON OTHER,TOTALLY UNRELATED NOTES: If you're looking for an intelligent movie in theaters now (always a challenging search), ARGO earns a qualified thumbs-up from me; my review is HERE. (If you find the review helpful, please click on "yes" at the bottom.) I've also just finished reading a thoroughly researched and well-written cultural biography of John Brown, Midnight Rising, that positions his raid on Harper's Ferry as a seminal moment in the lead-up to the Civil War. Tony Horwitz previously wrote Conservatives in the Attic, which -- as the descendant of Southerners -- I found spot-on.