New MMPI scale invalid as forensic lie detector, courts rule

Injured plaintiffs falsely branded malingerers?

Psychology's most widely used personality test, the MMPI, jumped into the national spotlight today in a fascinating David-and-Goliath controversy pitting corporate interests such as Halliburton against the proverbial little guy.

At issue is the "Fake Bad Scale" that was incorporated into the Minnesota Multiphasic Personality Inventory last year for use in personal injury litigation. A front-page critique in today's Wall Street Journal includes publication of the items on the contested scale, a test security breach that will no doubt have the publisher seeing red.

Although a majority of forensic neuropsychologists said in a recent survey that they use the scale, critics say it brands too many people - especially women - as liars. Research finding an unacceptably large false-positive rate includes a large-scale study by MMPI expert James Butcher, who found that the scale classified high percentages of bonafide psychiatric inpatients as fakers.

One possible reason for this is that the scale includes many items that people with true pain or trauma-induced disorders might endorse, such as "My sleep is fitful and disturbed" and "I have nightmares every few nights." Yet hearing the term "Fake Bad" will likely make a prejudicial impact on jurors even if they hear from opposing experts who say a plaintiff is not faking.

The controversy came to a head last year in two Florida courtrooms, where judges barred use of the scale after special hearings on its scientific validity. In a case being brought against a petroleum company, a judge ruled that there was "no hard medical science to support the use of this scale to predict truthfulness.” Other recent cases in which the scale has been contested include one against Halliburton brought by a former truck driver in Iraq.

The 43-item scale was developed by psychologist Paul Lees-Haley, who works mainly for defendants in personal injury cases and charges $600 an hour for his depositions and court appearances, according to the Journal article. In 1991, he paid to have an article supportive of the scale published in Psychological Reports, which the WSJ describes as "a small Montana-based medical journal."

The scale was not officially incorporated into the MMPI until last year, after a panel of experts convened by the University of Minnesota Press reported that it was supported by a "preponderance of the current literature." Critics maintain that the review process was biased: At least 10 of the 19 studies considered were done by Lees-Haley or other insurance defense psychologists, while 21 other studies – including Butcher's – were allegedly excluded from consideration.

Later last year, the American Psychological Association's committee on disabilities protested to the publisher that the scale had been added to the MMPI prematurely.

Lees-Haley, meanwhile, defends the scale as empirically validated and says criticism is being orchestrated by plaintiff's attorneys such as Dorothy Clay Sims, who has written guides on how to challenge the Fake Bad scale in court.

Even if the scale was valid before today, questions are certain to arise about the extent to which it will remain valid once litigants start studying for it by using today's publication of all 43 items along with the scoring key.

The lesson for forensic practitioners: Be aware of critical literature and controversy surrounding any test that you use in a forensic context, and be prepared to defend your use of the test in court.

The article, "Malingerer Test Roils Personal-Injury Law; 'Fake Bad Scale' Bars Real Victims, Its Critics Contend," which includes ample details on the controversy, is only available to Wall Street Journal subscribers, but you can try retrieving it with a Google news search using the term "MMPI Fake Bad." The University of Minnesota Press webpage on the contested scale is here, along with a list of research citations.

Here are citations to the major pro and con research articles:

"Meta-analysis of the MMPI-2 Fake Bad Scale: Utility in forensic practice," Nelson, Nathaniel W., Sweet, Jerry J., & Demakis, George J., Clinical Neuropsychologist, Vol 20(1), Feb 2006, pp. 39-58

"The construct validity of the Lees-Haley Fake Bad Scale: Does this measure somatic malingering and feigned emotional distress?: Butcher, James N., Arbisi, Paul A., & Atlis, Mera M., Archives of Clinical Neuropsychology, Vol 18(5), Jul 2003, pp. 473-485.

Postscript: Test distributor Pearson Assessments responded with alacrity - not to the heart of the controversy but to the Journal's reprinting of test items. The company, which makes a mint from selling and scoring the MMPI and other psychological tests,got the WSJ to remove the online link to the test items. In a "news flash," Pearson says it is "evaluating the impact of the article" and asks psychologists to report any other instances of "illegal" reproduction of the scale in publications, websites, chat rooms, or blogs.

NOTE: For more of my posts about the MMPI-2's Fake Bad Scale, search the blog using the term "MMPI" (the search box is in the upper left corner of the page).

Latest on controversial "Fake Bad Scale"

I wanted to alert my psychologist readers to the latest in the controversy over the "Fake Bad Scale" of the Minnesota Multiphasic Personality Inventory, a topic I have blogged about previously (HERE). If you are planning to use this Scale, you should be aware of this article and the others on both sides of the controversy.

The Fake Bad Scale (FBS) was developed to identify malingering of emotional distress among claimants in personal injury cases. It was recently added to MMPI-2 scoring materials, resulting in its widespread dissemination to clinicians who conduct psychological evaluations.

The latest article, in the interesting new journal Psychological Injury & Law, summarizes concerns about the Scale's reliability, validity, and potential bias against women, trauma victims, and people with disabilities.

The article concludes that the scale is not sufficiently reliable or valid to be used in court:

"Based on a review and a careful analysis of a large amount of published FBS research, the FBS does not appear to be a sufficiently reliable or valid test for measuring 'faking bad,' nor should it be used to impute the motivation to malinger in those reaching its variable and imprecise cutting scores. We agree with the conclusions of the three judges in Florida that the FBS does not meet the Frye standards of being scientifically sound and generally accepted in the field, and that expert testimony based on the scale should be excluded from consideration in court. The samples used to develop the FBS are not broadly representative of the populations evaluated by the MMPI-2, nor are its criteria used to define malingering objective and replicable. There is insufficient evidence of its psychometric reliability or validity, and there is no consensus about appropriate cut-off scores or use of norms."

The article is "Potential for Bias in MMPI-2 Assessments Using the Fake Bad Scale (FBS)." The Abstract and a "free preview" are online HERE; the full article requires a subscription but can be requested directly from the first author, James Butcher. Butcher and co-authors Carlton Gass, Edward Cumella, Zina Kally and Carolyn Williams present just one side of the heated controversy; a rebuttal is scheduled for publication in an upcoming issue of the journal, followed by other pro and con articles.

Related blog resources:

New MMPI scale invalid as forensic lie detector, courts rule: Injured plaintiffs falsely branded malingerers? (March 5, 2008) – contains links and citations to other sources

"Fake Bad Scale": Lawyers advocate exposing in court (May 20, 2008)

A list of FBS references and statement from the test's publisher is HERE

Hat tip: Ken Pope

"Fake Bad Scale": Lawyers advocate exposing in court

When a controversial test is being used against their client, attorneys may weigh the following questions:

• Should I seek an evidentiary hearing (under Frye or Daubert) and try to exclude the test?

• Or, should I let the test come in as evidence, and educate the jury about weaknesses in the underlying science?

This question regularly comes up at Sexually Violent Predator trials, regarding the controversial Static-99 risk assessment tool. Now, it is coming up in civil personal injury trials, regarding the MMPI-2's "Fake Bad Scale" (which I blogged about here back in March).

Increasingly, attorneys are choosing the second option when the science underlying a test is weak. They are openly critiquing the test and its findings, and allowing jurors to form their own conclusions. Yesterday's Lawyers USA features an article on how plaintiffs' attorneys are "turning the tables" on the Fake Bad Scale:

Although plaintiffs' attorneys are unanimous in despising the Fake Bad Scale, there is a mini-debate about whether it is more effective to exclude the test before trial or allow it in and discredit it while cross-examining the defense expert.

"It's a tough call, frankly," said Dorothy Clay Sims, a founding partner of Sims, McCarty, Amat & Stakenborg in Ocala, Fla., who has won three hearings over excluding the test.

"Frye and Daubert hearings are tough, but courts don't seem to like this test, so it's difficult to give up a hearing that you have a good chance of winning," she said. "On the other hand, once the Fake Bad Scale is demystified for the jury, and you pierce through it, they look at the defense doctor and say 'Oh, come on.' "

The article features the case of Sarah Jenkins, a medical receptionist who suffered tissue injuries and cognitive problems after her pick-up truck was hit by a delivery truck. She scored in the faking range on the Fake Bad Scale.

Rather than fighting to exclude the test, experienced trial attorney Dean Heiling made it a centerpiece. He cross-examined the defense expert at length about the test, and through his own expert exposed the controversy in the field about the test's validity.

Most interestingly, he put his client on the stand in rebuttal, and had her go through each test item and her answer with the jury.

Jurors deliberated only three hours before awarding a verdict of $225,749.

The lesson to forensic psychologists: Know your tests, and know their weaknesses.

The full story, by Sylvia Hsieh, is here, although it is only available to subscribers. For more on the controversy over the scale, see my previous post here.

Hat tip: Ken Pope

MMPI feud hits prime time

MMPI-2-RF version and Fake Bad Test at issue

The Minnesota Multiphasic Personality Inventory is the most widely used and best known personality test in the world. Daily, it is introduced in court in everything from child custody cases to civil lawsuits to criminal proceedings. Despite the fact that public tax monies sponsored its original development, it's become a major cash cow for the University of Minnesota, raking in about $1 million a year in royalties. But now, media focus on a bitter professional dispute is causing some in the field to wonder whether the legendary test has seen its day.

"Feud over famed test erupts at U," blared the headline on an in-depth investigation by Maura Lerner of the Minneapolis/St. Paul Star-Tribune.

At issue is last year's "dramatic makeover" of the highly profitable septuagenarian. The slimmed-down MMPI-2-RF (restructured form) has just 338 questions rather than the old test's 567.

Leading the critics is James Butcher, a retired psychologist whose career centered around the old MMPI-2. He claims the MMPI-2-RF is so radically altered that it amounts to a new instrument. "These folks have made a new test and they are using the name MMPI ... with all the 70 years of tradition to market [it]." If he's right, of course, then all of the myriad studies and normative data on the old test are irrelevant to interpreting scores on the MMPI-2-RF.

Of even more direct relevance to us forensic folks is the controversy surrounding the new "Fake Bad Scale." As I've blogged about previously (links below), the "FBS" is being used to brand personal injury claimants as malingerers. Critics say the 43-item scale "discriminates against women and is prone to 'false positives.' "

Defenders of the new test deride critics as "the Mult Cult" (a twist on Multiphasic). And, to be fair, some do have a vested interest in the previous edition: Butcher earns a 30 percent cut of the $600,000 annual royalties on the MMPI-2's computerized interpretation system.

All of the hoopla has led to a series of formal investigations. In one, a university audit revealed that most of the MMPI research grant money was going to projects involving advisory board members of University Press's test division, who in some cases had even reviewed their own grant proposals.

The Press's solution to the controversy, according to the Star Tribune, is to "let the marketplace decide."

Hmmm. Is the marketplace really the best judge of good science?

The hoopla is likely to benefit the MMPI's competitors. In forensic circles, the test's former monopoly hold is giving way as the Personality Assessment Inventory and other newer instruments gain ground.

Whatever side we practicing psychologists choose, the major test distributors won't care. They distribute them all, and they will continue to rake in enormous sums of money. Don't you just love those over-the-top shipping and handling fees charged by you-know-who?

Related blog posts:

• New MMPI scale invalid as forensic lie detector, courts rule: Injured plaintiffs falsely branded malingerers? (March 5, 2008)
• Latest on controversial "Fake Bad Scale" (February 23, 2009)
• "Fake Bad Scale": Lawyers advocate exposing in court (May 20, 2008)