Assessing “volitional control” in sex offenders

When I review government reports in sexually violent predator cases, I find that most focus on two things: (1) the person's risk of future sexual violence, and (2) whether that risk is related to a psychiatric disorder.

But this misses a critical piece of the puzzle. In order for a civil commitment based on future danger to be Constitutional under Kansas v. Crane, the former sex offender must also demonstrate a serious difficulty controlling his behavior.

It's understandable that some evaluators shy away from addressing this issue of so-called "volitional control." After all, it is not easy to measure. Far easier to assume a circular tautology, in which a failure to control one's behavior is advanced as evidence of inability to exert self control. But, as the American Psychiatric Association famously noted in a 1983 statement opposing conclusory opinions on volitional control in insanity cases:

"The line between an irresistible impulse and an impulse not resisted is probably no sharper than that between twilight and dusk."

Into this breach jumps psychologist Frederick Winsmann of Boston. In an article in the current issue of Sex Offender Treatment, he proposes a model for how to assess volitional control in sexually violent predator evaluations.

Credit: The Bad Chemicals

Winsmann theorizes that poor self control emanates from two related processes: (1) behavioral impulsivity, and (2) impaired decision-making. He recommends that evaluators incorporate screening measures that tap into these two processes, such as the Barratt Impulsiveness Scale and tests of executive (frontal lobe) functioning like the Wisconsin Card Sorting Test or the Iowa Gambling Task.

While this approach is a welcome step in the right direction, it must be recognized that tests of impulsivity and frontal lobe functioning are just indirect measures of the volitional impairment that is theorized to underlie some sexual offending.

Indeed, Winsmann stresses that these tests should be approached as part of a larger idiographic framework of understanding the person as a unique individual, and that poor test performance does not in and of itself establish volitional impairment. For example, scores may be lowered by poor cognitive abilities. (I have also seen cognitively normal people with fine self control do poorly on the Wisconsin Card Sorting Test due to high anxiety.)

The full article is available for free online (HERE).

SVP risk tools show 'disappointing' reliability in real-world use

Rater agreement on three instruments commonly used  to assess sex offenders' risk of recidivism is much lower in practice than reported in the tools' manuals, according to a new study out of Florida.

Faring most poorly was the Psychopathy Checklist (PCL-R). Correlations of scores between two evaluators hired by the same agency were in the low range. On average, psychologists differed by five points on the instrument, which has a score range of of zero to 40. In one case, two evaluators were apart by a whopping 24 points!

Agreement among evaluators was only moderate on the Static-99 and the MnSOST-R, two actuarial risk assessment instruments for which scoring is relatively more straightforward.

The study, published in the respected journal Psychological Assessment, was a collaboration between scholars from the Department of Mental Health Law and Policy at the University of South Florida and researchers with the Florida Department of Children and Families. It utilized archived records culled from the almost 35,000 individuals screened for possible Sexually Violent Predators (SVP) civil commitment in Florida between 1999 and 2009. The researchers located 315 cases in which the same individual was evaluated by separate clinicians who each administered both the PCL-R and at least one of the two actuarial measures within a short enough time frame to enable direct scoring comparisons.

It would be a mistake to lean too heavily on the results of a single isolated study. But the present study adds to a burgeoning body from several groups of independent researchers, all pointing to troubling problems with the accuracy of instruments designed to forecast risk of recidivism among sex offenders.

Related study: Psychopathy and sexual deviance not predictive

Collectively, the research has been especially critical of the ability of the highly prejudicial construct of psychopathy to add meaningfully to risk prediction in this high-stakes arena. Indeed, just this week another study has come out indicating that neither psychopathy scores nor sexual deviance measures improve on the accuracy provided by an actuarial instrument alone.

An especially interesting finding of that Canadian study is that reoffense rates were still below 12 percent over a 6-year followup period for even the most high-risk offenders -- those with high risk ratings on the Static-99R plus high levels of psychopathy and sexual deviance (as measured by phallometric testing). This makes it inappropriate to inflate risk estimates over and above those derived from Static-99R scores alone, the authors caution.

Item-level analysis finds varying rates of accuracy

A unique contribution of the Florida study is its analysis of the relative accuracy of every single item in each of the three instruments studied. Handy tables allow a forensic practitioner to see which items have the poorest reliability, meaning they should be viewed skeptically by forensic decision-makers.

For example, take the MnSOST-R, a now-defunct instrument with a score range of –14 to 31 points. The total gap between evaluators was as wide as 19 points; the items with the greatest variability in scoring were those pertaining to offenders' functioning during incarceration, such as participation in treatment.

Meanwhile, the weak performance of the Psychopathy Checklist owes much to the items on its so-called “Factor 1,” which attempt to measure the personality style of the psychopath. As I've discussed before, rating someone as “glib,” “callous” or “shallow” is a highly subjective enterprise that opens the door to a veritable avalanche of personal bias.

Piggy-backing off a recommendation by John Edens and colleagues, the Florida team suggests that the prejudicial deployment of the Psychopathy Checklist may be superfluous, in that scores on Factor 2 alone (the items reflecting a chronic criminal lifestyle) are more predictive of future violence or sexual recidivism.

Next up, we need to identify the causes of the poor interrater reliability for forensic risk prediction instruments in real-world settings. Is it due to inadequate training, differing clinical skills, variable access to collateral data, intentional or unintentional bias on the part of examiners, adversarial allegiance effects (not a factor in the present study, since both evaluators were appointed by the same agency), or some combination?

In the meantime, the fact that two evaluators working on the same side cannot reliably arrive at the same risk rating for any particular individual should certainly raise our skepticism about the validity of risk prediction based on these instruments.

The studies are:

Reliability of Risk Assessment Measures Used in Sexually Violent Predator Proceedings. Cailey Miller, Eva Kimonis, Randy Otto, Suzonne Kline and Adam Wasserman. Psychological Assessment. Published online 7 May 2012. Advance online publication. Click HERE to contact the authors.

Does Consideration of Psychopathy and Sexual Deviance Add to the Predictive Validity of the Static-99R? Jan Looman, Nicola A. C. Morphett and Jeff Abracen. International Journal of Offender Therapy and Comparative Criminology. Published online 28 May 2012. Click HERE to contact the authors.

Inter-rater reliability of the PCL-R total and factor scores among psychopathic sex offenders: Are personality features more prone to disagreement than behavioral features? John Edens, Marcus Boccaccini and D. W. Johnson. 2010. Behavioral Sciences and the Law, 28 (1), 106–119. 

Related blog posts: 

• Psychopathy: A Rorschach test for psychologists?
• Study: Psychopathy score fails to predict sexual recidivism   
• NPR series on psychopathy in court

Blogger wins scientific achievement award

Accepting the award. Photo credit: Michael Donner

I am pleased to report that I have been awarded the 2012 Distinguished Scientific Contribution in Psychology award. It struck like a thunderbolt in a clear blue sky; I had no idea I had even been nominated for an award until I got a phone call notifying me I had won. 

It was especially meaningful to come from the California Psychology Association. The only voice for California’s 18,000 licensed psychologists, the CPA tirelessly advocates for the profession as well as for the mental health needs of the general public in California.

For those of you who only know me as a blogger and/or a forensic psychology practitioner, I conducted pioneering research in the late 1990s into the motivations of hate crime perpetrators. I later extended that work to group rape, likening both forms of violence to cultural theater in which the actors publicly demonstrate masculinity, with their victims as dramatic props. (I'm excited about a forthcoming chapter in a cutting-edge text on multiple-perpetrator rape, due out next year.) I have also conducted historical research and published on the ethics of forensic diagnosis, and especially the contested sexual paraphilia of "hebephilia." More information on my research is available on my website and on Wikipedia.

The location of the awards ceremony could not have been more idyllic -- the gorgeous Monterey coast on a balmy weekend. The 270-degree view of the Monterey Bay and the surrounding hills from the 10th floor of the Marriott Hotel was breathtaking; unfortunately, a photo just can't capture it.

CPA President Craig Lareau presents award.
Photo credit: Patricia VanWoerkom

The quality of this year's convention trainings was impressive. Perhaps because the current president, Craig Lareau, is a forensic psychologist and attorney, there was a good deal of forensic programming. Alan Goldstein presented the latest on Miranda waiver evaluations (including the new instrument), Professor Gail Goodman gave an overview of the research on child witness accuracy, and there were workshops on forensic neuropsychology.

I especially enjoyed a presentation by Keely Kolmes of San Francisco and Heather Wittenberg of Maui designed to help psychologists step up their online presence. For anyone interested, Dr. Kolmes has some nice resources (HERE) for psychologists on the ethics of social media and on managing one's online reputation.

By the way, if you practice in California and don't belong to the CPA, I encourage you to join. The reconfigured CPA has a forward-looking leadership team headed by the dynamic Jo Linder-Crow and is doing essential advocacy work on behalf of psychologists and the public. It appears to have defeated (at least for the time being) an effort to axe our regulatory agency, the Board of Psychology, which would have left psychologists at the mercy of other professions. It's working hard to promote parity for mental health consumers. And it's tangibly supporting legislators who will lobby for progressive causes, for example prisoner rehabilitation instead of endless warehousing. So do your share, whether it's just paying dues or volunteering, so that all of the heavy lifting does not fall on just a few shoulders.

Sea Otter, Monterey Bay

Whether or not you belong to the CPA, if you are in California you might also consider donating to its Political Action Committee, which funds progressive politicians and reforms. The unfortunate reality is, politics is money-driven.

And now, sadly, it's back to the grindstone.

Related news: Your blogger profiled in the 2012 edition of advanced high school textbook, Forensic Science: Advanced Investigations.

Updates of leading malingering and Miranda rights tests

As we ring in the new year, here are a couple of updates on tests some of you may be using in your forensic practices.

Miranda instruments revised

When it comes to assessing whether a criminal defendant had the capacity to waive his or her Miranda rights prior to giving a statement to police, there is only one game in town -- the Instruments for Assessing Understanding and Appreciation of Miranda Rights by eminent forensic psychologist Tom Grisso and colleagues. Now, that set of tests has been completely revised and given a new name: the Miranda Rights Comprehension Instruments (MRCI). One of the main problems with the old instruments was that their language was more complex than the actual language used by many police warnings, reducing their real-world validity. That problem has been rectified through simpler language in the new edition. Also, the test developers promise better psychometric properties and updated normative data. But it's kind of a tricky situation, because we are being instructed to use the new instruments when assessing juveniles, but to stick with the old ones for assessing adults until new adult norms are published some time in 2013. More on the new instruments is HERE.

Facelift for gold-standard malingering test

In contrast to Miranda rights, there is no shortage of instruments for assessing malingering. But for assessing malingered psychosis, one instrument is widely recognized as the "gold standard." That test is Richard Rogers and colleagues’ Structured Interview of Reported Symptoms (SIRS). It too was recently revised. The revision was in part due to the discovery that the original SIRS failed to adequately distinguish between some people diagnosed with dissociative identity disorder and people who were instructed to deliberately exaggerate symptoms. But critical reception for the revised tool, the SIRS-2, has been less than stellar. Over at the Open Access Journal of Forensic Psychology, forensic psychologist Greg DeClue warns that the SIRS-2 may be vulnerable to admissibility challenges in court. That is due to the publishers' refusal to be transparent by letting independent professionals such as DeClue analyze their data. DeClue challenges the test's developers to submit a comprehensive description of the SIRS-2 validation study for peer-reviewed publication, and to let independent professionals analyze the data without impediment. His interesting critique is HERE.

Study: Psychopathy score fails to predict sexual recidivism

Many forensic psychologists believe that psychopathy is a risk factor for sex offender recidivism. Not surprisingly, when forensic psychologists assign a sex offender a high score on a psychopathy test, it increases the risk of extreme legal sanctions such as civil commitment.

But a new study out of Texas found zero correlation between sexual recidivism and psychopathy, as measured by the widely used Psychopathy Checklist (PCL-R). If anything, sex offenders who were arrested for a new sexually violent offense tended to have lower scores on the PCL-R than those who were not rearrested!

Regular blog readers should be familiar with these researchers by now: Dan Murrie, Marcus Boccaccini and crew are the same scholars who informed us of the partisan allegiance phenomenon, in which evaluators working for the government tend to assign far higher PCL-R scores than do those working for the defense.

In their most recent study, they examined PCL-R scores from about 400 sex offenders in Texas who were released from prison and then tracked for anywhere from about two to seven years. They examined not just the total scores on the PCL-R, but also scores on the instrument's two factors, as well as four so-called facet scores. Not one of these seven PCL-R variables was a statistically significant predictor of whether a man would be arrested for a new sex crime.

“Overall, these predictive validity findings were striking because the PCL-R apparently failed to predict the type of outcome (i.e., sexually violent reoffense) for which it was administered in this context,” the authors noted.

Further, in cases in which the PCL-R was administered by more than one evaluator, the study found poor agreement between the two, even though both were working for the government. Consistent with prior research, interrater agreement was higher on Factor 2, which measures antisocial behavior and an impulsive lifestyle, than on Factor 1, which measures the vaguely operationalized personality and emotional dynamics thought to underlie psychopathy.

In an interesting twist, the researchers tried to determine whether some evaluators were more accurate than others at predicting recidivism through PCL-R scores. They identified four highly prolific evaluators; together, these three psychologists and one medical doctor had provided almost two-thirds of the PCL-R scores in the study. Although the PCL-R scores of three of these four evaluators were more likely than other evaluators' scores to correlate with a new arrest for a non-sexual crime, even these evaluators could not produce PCL-R scores that predicted sexual offense recidivism.

Despite the PCL-R’s lack of predictive validity, sex offenders with higher PCL-R scores were more likely than others to be recommended for civil commitment, indicating that the unreliable rating was far from harmless in forensic practice.

The study is: 

Murrie, D. C., Boccaccini, M. T., Caperton, J. and Rufino, K. Field Validity of the Psychopathy Checklist–Revised in Sex Offender Risk Assessment. Psychological Assessment. Click HERE to request a copy from the first author, at the Institute of Law, Psychiatry, and Public Policy.

Of related interest:

• NPR expert panel: Weighing the value of a test for psychopaths
• Psychopathy: A Rorschach test for psychologists? (blog post)

More on test administration issues in Twilight Rapist case

Alan Cohen, the attorney in the Billy Joe Harris case that I blogged about last week, wrote to clarify the unusual test administration procedures of psychiatrist Colin Ross, who testified for the defense. Because his letter (which he posted at my Psychology Today blog, Witness) is of general interest to forensic psychology, I re-post it here, along with my response:

Mr. Cohen wrote:

I found your article of interest and hope this will create a forum for further discussion on DID and its use in the courtroom setting.

The issue of my administering the examination to my client took on a sinister spin from the way it was interpreted by Dr. [Robert] Barden when in fact it was nothing more then my hand-carrying it to the jail and passing the sealed envelope into the hand of a deputy who then gave it to my client. The transaction took less then a minute. I remained in an attorney booth with my client who spent four hours answering the self-administered questions. When he completed the exam he placed the results in an envelope and sealed it. He then handed the envelope to a deputy who then gave it to me. That transaction took less then a minute.

I personally carried the test to the jail so that the contents would not be examined by either the sheriffs department or the prosecutors office since Mr. Harris was under extremely tight surveillance and the results of the test would/could form the basis of our defense. I could not jeopardize the results of the exam being compromised by falling into the "wrong hands."

* * * * *

Mr. Cohen,

Thanks for writing to clarify the circumstances of the test administration. I have seen other cases in which psychologists have had third parties administer psychological tests, or have even given prisoners tests to fill out in their spare time and return at their leisure. While the intermediary who delivers the test is not doing anything sinister, from the standpoint of professional ethics and practice there are several problems with such practices.

First and foremost, if a test is standardized -- that is, if it has norms to which an individual is being compared -- then such procedures violate the standardized administration and may invalidate the results.

Second, such procedures violate test security.

Third, they prevent the expert from ensuring the adequacy of testing conditions, or of observing the individual as he performs the tasks; observation by skilled examiners can be an important component of one's ultimate opinions. Relatedly, sitting with the test-taker allows the examiner to assess for adequate comprehension, and answer any questions that may come up.

When Dr. Barden testified that it was unethical for the attorney to administer the tests, he was likely referring to the Ethics Code for psychologists, as well as the Standards for Educational and Psychological Testing ("The Standards") promulgated by the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education.

As noted in the introduction to the Standards, which apply to everyone who administers, scores and interprets psychological or educational tests, regardless of whether they are a psychologist:

The improper use of tests can cause considerable harm to test takers and other parties affected by test-based decisions. The intent of the Standards is to promote the sound and ethical use of tests and to provide a basis for evaluating the quality of testing practices. 

Collectively, the Ethics Code and the Standards require that:

• Test administrators receive proper training (Ethics Code 9.07; Standards 12.8)
• Tests not be administered by unqualified persons (Ethics Code 9.07; Standards 12.8)
• Examinees receive proper informed consent (Ethics Code 9.03; Standards 12.10)
• Test data be kept confidential and secure (Ethics Code 9.04; Standards 12.11)
• Assessment techniques be protected from disclosure to the extent permitted by law (Ethics Code 9.11; Standards 12.11) 

Again, I appreciate your taking the time to write.

NOTE: After I posted this exchange, the testifying psychiatrist, Colin A. Ross, posted a comment at my Psychology Today blog. He provided more information about the screening tests for dissociation and why they were administered as they were. He also offered his opinion on the validity of Dissociative Identity Disorder. His comment can be viewed HERE. Please feel free to join in the discussion, either here or (preferably) at my Witness blog, where the conversation began.

Multiple personality excluded in Texas insanity case

A serial rapist’s attempt to claim insanity based on multiple personality disorder fell flat, as a judge ordered the expert's trial testimony stricken from the record as junk science.

Billy Joe Harris

Psychiatrist Colin Ross testified that Billy Joe Harris, the so-called "Twilight Rapist" who targeted elderly women, suffered from multiple personality disorder -- now known as dissociative identity disorder (DID) -- brought on by childhood abuse.

Ross, who runs the Colin A. Ross Institute that provides trainings on psychological trauma and dissociative identity disorder, testified that the condition’s presence in the Diagnostic and Statistical Manual of Mental Disorders (DSM) published by the American Psychiatric Association establishes it as a "real and valid disorder."

Ross testified that he gave the defendant three tests for DID. However, in a most unusual procedure, rather than personally administering the tests, he gave them to the defense attorney to administer. Thus, he has no way of knowing for sure who filled in the tests, or under what circumstances. 

Ross testified that the defendant's scores on a screening test, the Dissociative Experiences Scale, were so high that he questioned the test's validity. He also conceded that the defendant was "clearly telling stories that are not true" about other aspects of his life, for example falsely claiming to have served in Iraq when he was actually in Saudi Arabia. However, Ross testified that after getting a chance to talk personally with one of Harris's alters, "Bobby," he was convinced of Harris's claim of multiple personalities.

"I don't think he's faking the dissociative identity disorder," he testified. "I could be wrong."

The real culprit, David the Dog

The defendant, a former prison employee, also took the witness stand, "weaving tales of bestiality, aliens, transvestites and combat heroism," in the words of news reporter Sonny Long. Harris testified that he had three other personalities inside him, including a black Great Dane named David who committed the rapes.

A dramatic moment came during cross-examination, when prosecutor Bobby Bell asked to speak to the defendant's alter, also named Bobby. As Long described the scene:

Harris lowered his head momentarily, raised it back up, rolled his neck and declared in a deep voice to be "Bobby."

Several jurors stifled laughter during the subsequent give-and-take between Harris and Bell, according to Long's account in the Victoria Advocate.

But perhaps even more damaging to Harris's credibility was an audiotape played for the jury in which he talks to his girlfriend about having put on "a good show" in court one day. Earlier that day, he had fallen to the floor and twitched and shook until he was restrained. The girlfriend warned Harris that the telephone call was being recorded, to which Harris replied, "I know it."

Forensic psychologist Walter Quijano also testified for the defense. (If the name sounds familiar, he has been in the spotlight for using race as a risk factor in death penalty cases, as I recently blogged about.) He testified that when multiple personality popped up as an issue, he stepped back because that is not an area of expertise for him. However, he did testify that it is unusual for someone to begin a rape career so late in life. Harris is 54.

Mere presence in DSM doesn’t establish validity

After the defense rested, the prosecution called as a rebuttal witness a Minnesota psychologist and attorney who has made a crusade out of pushing so-called "junk science" out of the courts.

Robert Christopher Barden testified that dissociative identity disorder (aka multiple personality disorder) is a controversial condition looked upon with skepticism by the scientific mainstream. He cited several articles rejecting the condition as a viable diagnosis, despite its presence in the DSM.

"Because something is in the DSM doesn't mean it's reliable or should be allowed in a court of law," he testified, according to an article in the Victoria Advocate. "One of the ways to get junk science out of the legal system is you rely on the relevant scientific community. If something is controversial it means it's not generally acceptable."

Barden said the number of mental health professionals who tout dissociative identity disorder as viable are few and far between.

"There are a few pockets of people left who are doing this," he said. "The scientists I know condemn it to be the worst kind of junk science and dangerous to the public. Controversial and experimental theories should not be allowed to contaminate the legal system."

Concerning the tests given to Harris, Barden said, "There's no magic to these tests. It looks scientific. It looks professional, but when you get down into it, it's junk. It's unusual for a psychiatrist to interpret a psychological test and it's highly unethical for Mr. Cohen [the defense attorney] to give the tests."

After Barden’s testimony that the condition is not generally accepted by the scientific community, despite the fact that it is listed in the DSM, District Judge Skipper Koetter ordered Dr. Ross’s testimony on dissociative identity disorder stricken from the record.

Justice, Texas-style

In the end, the defendant’s overdramatization and courtroom theatrics likely did him in. During the trial, he trembled and twitched and sat in the courtroom with paper stuffed in his ears, which his attorney said was “to keep the voices from speaking to him."

The jury took only 10 minutes to convict Harris, and another 10 minutes later in the month to sentence him to life in prison.

After the verdict, Barden said in a press release that the outcome demonstrates “the power of science-law teams in protecting the legal system from junk science testimony."

Barden has been involved in hundreds of lawsuits, criminal prosecutions and licensure actions across the United States over the past two decades, targeting not only multiple personality disorder but also quack therapists in the repressed memory and rebirthing therapy movements.

Judge Koetter's ruling is not the last word, of course, as it is just one trial judge's opinion. Appellate courts in other states have ruled differently. For example, in the 1999 case of State v. Greene (139 Wn. 2d 64), the Washington Supreme Court held that dissociative identity disorder was a generally accepted diagnosis because it was listed in the DSM-IV, and therefore met the Frye test for admissibility. But the Court went on to say that the applicability of this diagnosis to the issue of criminal responsibility was problematic and that testimony about DID was not "helpful" to the jury. (The Trowbridge Foundation has more information on this case HERE.)

The battle lines over dissociative identity disorder have heated up in the dozen years since that ruling, so who knows how an appellate court might rule today.

For those interested in learning more about the controversy, I recommend the chapter "Dissociative Identity Disorder: Multiple Personalities, Multiple Controversies" by Scott Lilienfeld and Steven Jay Lynn, in their book, Science and Pseudoscience in Clinical Psychology.

Psychopathy: A Rorschach test for psychologists?

• Compassion
• Empathy
• Impulsivity
• Excitement-seeking

What do these personality traits have in common?

If you are high on any or all of them, you may be less likely to rate other people as psychopathic on the Psychopathy Checklist (PCL-R).

The PCL-R is the most widely used measure of psychopathy in the world. But in real-world forensic settings, scores vary widely depending upon which side retained the evaluator. This finding is called the "partisan allegiance" effect.

In a new twist, these same researchers that brought you partisan allegiance have found that an evaluator's personality may impact her judgments of psychopathy. Evaluators low on compassion and thrill-seeking as measured by a widely used personality test, the NEO Personality Inventory-Revised, are more likely than others to rate criminals as psychopathic.

That’s ironic, because according to the theory of psychopathy, it's supposed to be the psychopath -- not the psychologist -- who has a deficit in empathy.

The exploratory study, forthcoming in the journal Assessment, was based on a small sample of 22 individuals who were given nine hours of training by a clinical psychologist with substantial research and forensic practice experience with the PCL-R. "The daylong session was an attempt to replicate typical PCL-R training procedures," the study authors explain.

The researchers emphasize that their findings are preliminary and need to be replicated and extended. But if they hold up, they have intriguing implications not only for the psychopathy measure but also for other psychological tests with elements of subjectivity in scoring or interpretation.

The study did not examine the accuracy of the low versus high scorers. But if low-scoring evaluators are more empathetic, this implies that they may be more accurate in interpersonal assessment contexts.  

Subterranean class conflict?

Future research might examine class background, race and philosophical  beliefs to see if these influence scoring of the Psychopathy Checklist. In my informal observations, professionals who look for psychopaths under every rock tend to lack understanding of, or empathy for, those on the bottom.

Here's how that looks in practice:

The upper middle-class professional walks into the evaluation room, oblivious to the blinders and unconscious biases she brings to the table. Her subject, in contrast, is far from oblivious. With his more acute empathetic skills, the lower-class or minority individual accurately reads the professional's bias against him, which she transmits through nonverbal and other deniable cues. He also realizes that she holds all the power, and that her judgments will affect his future in very tangible ways.

He reacts with defensiveness, suspicion, or muted hostility -- especially if she is working for "the other side." But not recognizing his reaction as part of an interactional dance that she herself set in motion, the evaluator interprets his stance as evidence of intrinsic personality defect. She may see him as glib, superficially charming, conning, or manipulative -- all facets of Factor 1 (the personality dimension) on the Psychopathy Checklist.

In this interaction, all of the power belongs to the person who gets to do the labeling. Scoring and labeling the offender becomes a circular process through which the evaluator -- especially when primed by adversarial allegiance -- can  project her own class- or race-based prejudices, distancing herself from the evil other, while at the same time denying complicity. An obfuscating tautology is proffered as a simple explanation for complex and multi-determined antisocial acts.

There is more to the construct of psychopathy, of course. I focus here on its potential subjectivity because this is a facet that proponents rarely acknowledge, especially in public. Forensic experts owe a duty to explain the subjectivity of the PCL-R when it is introduced in court, where the label "psychopath" can be the kiss of death. When labeled as psychopaths:

• Juveniles are more likely to face harsh punishment in adult court.
• Sex offenders are more likely to be preventively detained.
• Capital murder defendants are more likely to receive the death penalty.

So, the next time a promising young student proposes to study psychopathy or "the criminal mind," you might give her a gentle nudge in a more fruitful direction: Rather than treading this tired old path, she might contribute more to the field by studying the psyches of professionals who assign such diagnostic labels in the first place. 

• The study is: On Individual Differences in Person Perception: Raters' Personality Traits Relate to Their Psychopathy Checklist -Revised Scoring Tendencies. The authors are Audrey K. Miller, Katrina A. Rufino, Marcus T. Boccaccini, Rebecca L. Jackson and Daniel C. Murrie. Assessment, forthcoming.

• Of related interest: Social Class, Contextualism, and Empathic Accuracy, by Michael W. Kraus, Stéphane Côté and Dacher Keltner, Psychological Science, November 2010.

Child abuse assessment: Special issue

Forensic psychologist Eric Mart has guest edited a special issue of the Journal of Psychiatry and the Law on assessment and testimony in cases of child abuse. The articles address both scientific and practical aspects of child abuse assessment, testimony, and research. They include:

Maternal Filicide and Mental Illness in Italy: This retrospective review co-authored by Geoffrey McKee, who has written books on filicide, and Alesandra Bramante compares the forensic characteristics of mothers with and without severe mental illness who killed their children.

Interviewing Immigrant Children for Suspected Child Maltreatment: Lisa Aronson Fontes examines challenges posed in forensic interviews of immigrant children when there is a suspicion that these children may be victims of child abuse or neglect. Suggestions are made
for interviewers regarding the interview setting, preparations, building rapport, conveying respect, narrative training, pacing the interview, and trauma symptoms that may stem from issues unrelated to the abuse.

Persistent Problems with the "Separation Test" in Munchausen Syndrome by Proxy: Munchausen syndrome by proxy remains a controversial diagnosis because information is easily tainted to make the mother appear responsible for her child’s symptoms. Loren Pankratz critiques (and offers alternatives to) the “separation test,” a scientifically problematic procedure that is often used to gather evidence against the mother.

Common Errors in the Assessment of Allegations of Child Sexual Abuse: Finally, the special issue editor himself tackles common errors in the complex, challenging, and high-stakes undertaking of assessing CSA allegations. After reviewing frequent causes of substandard investigations, the illustrious Dr. Mart provides ideas for research-based methods to improve the situation.

Blogger challenges my post on sex offender risk assessment

My report on a promising new method for assessing sex offender recidivism risk prompted a reaction from Robin Wilson over at a new blog by the publishers of Sexual Abuse, the journal of the Association for the Treatment of Sexual Abusers (ATSA).

My interest dissolved into disappointment when I realized that the response sidestepped substantive discussion of the new research, a collaborative project by scholars in the United States, New Zealand, and Australia finding that accuracy of risk prediction can be enhanced by using age-stratified tables.

Instead of engaging with the scientific merits of this study, Dr. Wilson focused on the Static-99 actuarial tool, chastising me for being overly critical of that approach:

Dr. Franklin has something of a penchant for the flamboyant. Actually, I often appreciate a bit of bluster, but I think we also need to avoid bias and unnecessary sideswipes at our colleagues. In the spirit of honesty, let me say that I work in a civil commitment center and that I am a certified trainer for the Static-99 group of measures. So, my potential bias is on the table.

Is having an opinion a form of bias?

The term bias refers to a partiality that interferes with the ability to be fair or objective. Expressing a point of view does not make one biased. Rather, the articulation of informed opinion advances professional practice. Indeed, such "expert opinion" is precisely what courts seek from forensic psychologists when they solicit our testimony. An expert witness is expected to have a firm grounding in the applicable science, and to be able to critically analyze its strengths and weaknesses as it applies to a specific issue or case.

It is that process of reflection -- invoking complementary theoretical perspectives from criminology, sociology, anthropology and international studies relevant to forensic psychology -- that germinates many an idea for this blog. I welcome critical engagement with these ideas, but hope that interlocutors can themselves refrain from diversionary ad hominem arguments.

Why pick on the Static-99?

Dr. Wilson's main theme is that I am unduly critical of the Static-99 and its developers.

The Static-99 website promotes the tool as "the most widely used sex offender risk assessment instrument in the world." As Dr. Wilson himself notes in another publication, its "prolific use in sexual offender civil commitment proceedings" is one reason it generates so much debate.

But Dr. Wilson is correct in pointing out that many of the Static-99's flaws apply equally to its cousins. In referencing the Static-99, I in no way intend to exempt other actuarials from critique. Indeed, in previous posts I have discussed problems with the MnSOST-R and have reported the results of studies on the relative accuracy of the Static-2002, Risk Matrix 2000, RRASOR and other actuarial tools.

Dr. Wilson makes an essential point in defense of the Static-99. By encouraging us to anchor our risk estimates in empirical data, he reminds us, actuarials serve as a counterbalance against clinicians' inherent tendency to vastly overestimate sex offender risk:

Maybe, what Dr. Franklin should be highlighting more is the fact that some evaluators are falling prey to partisan concerns. A well done sex offender risk assessment [using an actuarial instrument] should say the same thing, regardless of who you are working for.

In principle, Dr. Wilson is entirely correct. Sadly, from my vantage point as a forensic psychologist working in the courtroom trenches, I see endemic partisan allegiance in this arena. The Static-99 is being systematically deployed to provide a scientific veneer for highly questionable practices in the service of civil commitment.

The problem is not just a handful of rogue evaluators. The latest permutation of the Static-99 fosters such misuse. I am referring to the developers' advice to assign sex offenders to one of four categories with different levels of actuarial risk. As I pointed out in a previous post, this procedure introduces a large element of clinical judgment into a procedure whose very existence is predicated on doing away with such subjectivity. In the polarized Sexually Violent Predator arena, evaluators now have an easy way to elevate their estimate of an offender's risk by comparing the individual to the highest-risk group rather than to the lower recidivism figures for sex offenders overall. This practice reflects a tautology, or logical fallacy. In a civil commitment proceeding the evaluator essentially says: "This offender is high risk. Just look at the recidivism rates for offenders in the high-risk group to which I have assigned him."

The Static-99 developers may be quite well intentioned, as Dr. Wilson says. But subjective intent is far less important than real-life effects. The road to Hell, as they say, is paved with good intentions. I will not be alone in cheering when I hear that the Static-99 developers have stepped up and publicly condemned the systematic misuse of their instruments in courthouses all across the United States.

If Dr. Wilson's point was to chastise me for the use of the term reliability where I meant to say accuracy, I stand corrected. (I have made the correction to the original post.) However, I make no apologies for expressing an informed viewpoint and, indeed, I unflinchingly embrace such a role as part of my professional obligation. The goal of my blog is not to convince people that I have all the answers. Rather, I am happy if I encourage critical reflection and stimulate people to read the original source material. In so doing, I hope I am doing my part to advance science and combat bias.

The award-winning Texas blog Grits for Breakfast has a more favorable reaction, with links to additional topics of interest.

Paradoxical finding on juvenile sex offender risk

Leading tools cannot distinguish among delinquent boys

New tools designed to predict sex offender recidivism risk among juveniles continue to generate controversy and confusion. Some studies find that they work a bit better than the flip of a coin, and some find that they don't. Now, a new study out of Canada adds a paradoxical twist to the mix.

On two leading instruments, generally delinquent juveniles with a prior sex offense scored higher in risk than juveniles who had committed only sex offenses. And, true to this prediction, about 13 percent of these delinquent boys went on to commit another sex offense during the 6.6-year followup period, compared with just 7 percent of the boys with only sex offenses.

But, while they did moderately well at predicting risk among the non-delinquent juveniles, neither tool could reliably predict which among the delinquent, higher-risk youngsters would go on to reoffend sexually. The two instruments were the Juvenile Sex Offender Assessment Protocol-II (J-SOAP-II) and the Estimate of Risk of Adolescent Sexual Offense Recidivism (ERASOR).

A number of factors contribute to the difficulties in accurately predicting risk, especially among juveniles. These include:

• Adolescent immaturity -- most juvenile offending is time-limited, and will spontaneously cease over time.

• Low base rates -- it is hard to accurately predict events that have a low likelihood of occurring. In the current study, for example, only 9.4 percent of the youths were charged with another sex offense during the followup period.

• Situational and random influences -- A lot of offending, especially among juveniles, is due to situational and environmental factors rather than personality variables, and these are extraordinarily hard to predict. (See this PSYBLOG post for a social psychology experiment demonstrating the underappreciated influence of situational variables.)
  

It all gets back to the larger problem I discussed in last week's report on the British metaanalysis of violence risk prediction tools: Maybe we will never find the Holy Grail. Maybe we've reached the summit of the mountain, and it’s time to step back.

The new study, by Gordana Rajlic and Heather M. Gretton of the Youth Forensic Psychiatric Services in Burnaby, British Columbia, is "An Examination of Two Sexual Recidivism Risk Measures in Adolescent Offenders: The Moderating Effect of Offender Type." It was published in the latest issue of Criminal Justice and Behavior and is available for free from Sage Publications for a limited time.

Related blog resources:

• New study on juvenile sex offender treatment: Efficacy claimed, but control group questionable (June 18, 2010)

• The "juvenile sex offender": Myth in the making? Book describes harmful effects of labeling and treatment (Jan. 31, 2010)

• New findings on juvenile sex offending (Jan. 7, 2010)

• Can we tell which juveniles will sexually reoffend? (Dec. 2, 2009)

• Challenge to juvenile sex offender risk prediction: Harsh federal law on shaky scientific ground (Oct. 9, 2008)

Forensic update: Malingering and incompetency

New response style measure hitting the market

Randy Otto and colleagues are as busy as beavers these days. I just blogged about Dr. Otto's new violence risk assessment text; now he and colleagues Jeffrey Musick and Christina Sherrod are releasing the Inventory of Legal Knowledge, a brief measure of response style for defendants undergoing adjudicative competence evaluations.

The 61-item, true-false test is orally administered and takes about 15 minutes. Questions are phrased in simple language and concern the legal process. It's highly portable for in-custody settings. And at $129 for the complete kit it's not a bad deal, considering PAR's normally hefty pricing. (You can order it now, but it won't actually release until mid-September.)

The theoretical basis is the same as for the Test of Memory Malingering and other measures of malingering; one suspects dissimulation when scores are either significantly lower than chance or significantly lower than attained by relevant normative groups.

A validation study has been published online in advance of the print edition of Assessment. The study, by the test authors, is titled: “Convergent validity of a screening measure designed to identify defendants feigning knowledge deficits related to competence to stand trial.” Here is the Abstract:

Because some defendants undergoing evaluation of their competence to stand trial may feign limitations in their ability to understand and participate in the legal process, assessment of their response style is critical. Preliminary research indicates that the Inventory of Legal Knowledge (ILK) has some potential to identify persons feigning competence related impairments. This study examined the convergent validity of the ILK using a sample of criminal defendants who, while undergoing competency evaluations, were administered the ILK and other response style measures. Moderate correlations between the ILK and these other tools provided some support for the ILK as a measure of response style.

More information is HERE.

Online review of updated SIRS malingering test

Richard Rogers' widely utilized Structured Inventory of Reported Symptoms (SIRS) has just been updated with a second edition, and forensic psychologist Steve Rubenzer of Houston, Texas (where "we don't only sing, but we dance just as good as we walk") has a critical review in the Open Access Journal of Forensic Psychology. The abstract:

The Structured Inventory of Reported Symptoms-2 (SIRS-2) contains significant changes designed to prevent false-positive and false-negative classification errors. While the SIRS-2 has many laudatory features, the manual contains some erroneous and questionable statistics and arguments, and authors sometimes stray from the best practices advocated by the first author. The SIRS-2 is a strong choice for assessing feigned psychosis and severe psychopathology. However, evidence for its value in assessing many other conditions, particularly somatic complaints and feigned cognitive impairment, is quite limited.

The full review is online HERE.

Overview: Feigning adjudicative incompetence

And, there's more! Sherif Soliman and Phillip J. Resnick have an overview of accepted methods for detecting feigning in competency evaluations, including specific instruments and their limitations, forthcoming from in an upcoming issue of Behavioral Sciences & the Law that is now online in advance of the print edition. The abstract:

Competence to stand trial (adjudicative competence) is the most requested forensic psychiatric evaluation, with an estimated 60,000 referrals annually. The challenge of detecting feigned incompetence has not been systematically studied until the past decade. Estimates of feigned adjudicative incompetence vary from 8 to 21%. This article reviews techniques for detecting malingered psychosis and malingered cognitive impairment during competence evaluations. Specific techniques for assessing feigned adjudicative incompetence and estimating the malingerer's genuine abilities are discussed. A stepwise approach to suspected feigned adjudicative incompetence is proffered.

Since this manuscript was already in press when the new ILK and the revised SIRS-2 were released, these instruments are not reviewed. In addition to a basic overview of instruments, however, Soliman and Resnick also describe a structured approach. As the authors point out, "Unlike many other evaluations, the assessment of adjudicative competence does not end with the determination that the defendant is malingering." Even with an uncooperative individual, the evaluator must make an effort to determine the subject's genuine capabilities.

Correspondence about that article should go to Dr. Soliman, Senior Clinical Instructor of Psychiatry at Case Western Reserve University in Ohio.

Response bias: Faith or science?

Most extensively studied topic in applied psychological measurement

After one hundred years and thousands of research studies, perhaps we are no closer than ever to understanding how response bias -- a test-taker's overly positive or negative self-presentation -- affects psychological testing. Perhaps what we think we know -– especially in the forensic context -– is mostly based on faith and speculation, with little real-life evidence.

That is the sure-to-be controversial conclusion of a landmark analysis by Robert E. McGrath of Fairleigh Dickinson University, an expert on test measurement issues, and colleagues. The dryly titled "Evidence for response bias as a source of error variance in applied assessment," published in Psychological Bulletin, issues a challenge to those who believe the validity of testing bias indicators has been established, especially in the forensic arena.

The authors conducted an exhaustive literature review, sifting through about 4,000 potential studies, in search of research on the real-world validity of measures of test response bias. They sought studies that examined whether response bias indicators actually did what we think they do -- suppress or moderate scores on the substantive tests being administered. They searched high and low across five testing contexts -- personality assessment, workplace testing, emotional disorders, disability evaluations, and forensic settings. Surprisingly, out of the initial mountain of candidate research, they found only 41 such applied studies.

Of relevance here, not a single study could be found that tested the validity of response bias indicators in real-world child custody or criminal court proceedings. Indeed, only one study specifically targeting a forensic population met inclusion criteria. That was a 2006 study by John Edens and Mark Ruiz in Psychological Assessment looking at the relationship between institutional misconduct and defensive responding on test validity scales.

Does the "Response Bias Hypothesis" hold water?

The authors tested what they labeled the response bias hypothesis, namely, the presumption that using a valid measure of response bias enhances the predictive accuracy of a valid substantive indicator (think of the K correction on the MMPI personality test). Across all five contexts, "the evidence was simply insufficient" to support that widely accepted belief.

McGrath and colleagues theorize that biased responding may be a more complex and subtle phenomenon than most measures are capable of gauging. This might explain why the procedure used in typical quick-and-dirty research studies -- round up a bunch of college kids and tell them to either fake or deny impairment in exchange for psych 101 credits -- doesn't translate into the real world, where more subtle factors such as religiosity or type of job application can affect response styles.

It is also possible, they say, that clinical lore has wildly exaggerated base rates of dishonest responding, which may be rarer than commonly believed. They cite evidence calling into question clinicians' widespread beliefs that both chronic pain patients and veterans seeking disability for posttraumatic stress disorder are highly inclined toward symptom exaggeration.

Unless and until measures of response bias are proven to work in applied settings, using them is problematic, the authors assert. In particular, courts may frown upon use of such instruments due to their apparent bias against members of racial and cultural minorities. For example, use of response bias indicators has been found to disproportionately eliminate otherwise qualified minority candidates from job consideration, due to their higher scores on positive impression management. (Such a finding is not surprising, given Claude Steele's work on the pervasive effects of stereotype threat.)

"What is troubling about the failure to find consistent support for bias indicators is the extent to which they are regularly used in high-stakes circumstances, such as employee selection or hearings to evaluate competence to stand trial and sanity," the authors conclude. "The research implications of this review are straightforward: Proponents of the evaluation of bias in applied settings have some obligation to demonstrate that their methods are justified, using optimal statistical techniques for that purpose…. [R]egardless of all the journal space devoted to the discussion of response bias, the case remains open whether bias indicators are of sufficient utility to justify their use in applied settings to detect misrepresentation."

This is a must-read article that challenges dominant beliefs and practices in forensic psychological assessment.

Psychopathy controversy goes primetime

More than a million people worldwide will get a chance to learn about psychology's internal controversy over psychopathy tomorrow, when Science publishes an article on the censorship allegations that I blogged about May 30.

Perhaps not coincidentally, just as the June 11 issue of the world's leading scientific news outlet hits the presses, the American Psychological Association is suddenly publishing the disputed article that was siderailed for more than three years.

Forensic psychologists Jennifer Skeem and David Cooke submitted the contested article to Psychological Assessment in 2006. It was peer reviewed, accepted, and scheduled for publication in 2007, but was derailed after Robert Hare, inventor of the Psychopathy Checklist (PCL), threatened to sue for defamation.

As you will remember from my previous blog post, the controversy surfaced in an opinion piece last month in the International Journal of Forensic Mental Health by two psychology-law leaders.

"[T]he threat of litigation constitutes a serious threat to academic freedom and potentially to scientific progress," wrote attorney John Petrila and psychologist Norman Poythress. "Academic freedom rests on the premise that advances in science can only occur if scholars are permitted to pursue free competition among ideas. This assumes that scholars have the liberty to do their work free from limitations imposed by political or religious pressure or by economic reprisals."

Hare now says he is "upset colleagues are suggesting he squelched academic debate," Science writer John Tavris reports, as his "lawsuit threat was meant only to get the 'attention' of APA, Skeem, and Cooke and force changes to the article."

The Science report is a sidebar to a larger piece on reform efforts over plaintiff-friendly libel laws in the United Kingdom. That country's laws, in which the defendant bears the burden of proof, are under fire from around the world over their allegedly chilling effect on scientific research on controversial topics. Critics say they encourage "libel tourism," in which corporations sue there over alleged offenses that occurred elsewhere.

PCL-R reification hampering science

The contested article by Skeem and Cooke, "Is Criminal Behavior a Central Component of Psychopathy? Conceptual Directions for Resolving the Debate," posits that the field of forensic psychology has prematurely embraced Hare's Psychopathy Checklist-Revised (PCL-R) as the gold standard for psychopathy, due in large part to legal demands for a tool to predict violence. Yet the PCL-R's ability to predict violent recidivism owes in large part to its conflation of the supposed personality construct of psychopathy with past criminal behavior, they argue:

[T]he modern justice context has created a strong demand for identifying bad, dangerous people…. [The] link between the PCL and violence has supported a myth that emotionally detached psychopaths callously use violence to achieve control over and exploit others. As far as the PCL is concerned, this notion rests on virtually no empirical support…. [T]he process of understanding psychopathy must be separated from the enterprise of predicting violence.

Criminal behavior weighs heavily in the PCL's 20 items because the instrument emerged from research with prisoners. But using the PCL-R's consequent ability to predict violence to assert the theoretical validity of its underlying personality construct is a tautological, or circular, argument, claim Skeem and Cooke. Or, as John Ellard put it more directly back in 1998:

"Why has this man done these terrible things? Because he is a psychopath. And how do you know that he is a psychopath? Because he has done these terrible things."

Rebuttal and response

Alongside the critique, Psychological Assessment has published a rebuttal by Robert Hare and Craig Neumann, along with a surrebuttal by Cooke and Skeem. Hare and Neumann accuse the critics of erecting a straw-man argument and misrepresenting their work:

The very title of their article is a straw man based on the unfounded claim that Hare and his colleagues consider criminality to be central or fundamental to the psychopathy construct. Their claim is bolstered by arguments misconstruing our published work and that of others and by quotes of our work that have been taken out of context or reconstructed in such a way that it appears that we have said something that we did not say. Skeem and Cooke also made highly selective use of the literature, often omitting published studies that directly contradict or do not support the points they attempted to make, particularly with respect to the role of antisocial tendencies in clinical and empirical conceptions of psychopathy. These tactics are inconsistent with their tutorial on the philosophy of science, compromise their arguments, and divert attention from any legitimate issues raised in their article. We contend that Skeem and Cooke did the field a disservice by presenting an inaccurate account of the role of the PCL–R in theory and research on psychopathy, both applied and basic.

I encourage readers to analyze all three papers, along with the two reports in Science, and draw your own conclusions.

• My original post on the controversy is HERE; a brief followup is HERE.
• Science writer John Travis' blog post on the story, in Science Insider (with more links), is HERE.
• Coverage at Mind Hacks is HERE.
• For more background and links on the UK libel law reform campaign, see my March 27 post, "Lie detection critic silenced by lawsuit."
  

The current issue of Psychological Assessment contains another article pertaining to the controversial psychopathy construct. In their abstract of "Validity of Rorschach Inkblot scores for discriminating psychopaths from nonpsychopaths in forensic populations: A meta-analysis," authors James Wood, Scott Lilienfeld and colleagues assert:

Gacono and Meloy (2009) have concluded that the Rorschach Inkblot Test is a sensitive instrument with which to discriminate psychopaths from nonpsychopaths. We examined the association of psychopathy with 37 Rorschach variables in a meta-analytic review of 173 validity coefficients derived from 22 studies comprising 780 forensic participants…. The present findings contradict the view that the Rorschach is a clinically sensitive instrument for discriminating psychopaths from nonpsychopaths.