Age tables improve sex offender risk estimates

First, how old is the bus driver?

If 30 people are riding on a bus, and 30 more people board the bus, how old is the bus driver?

The answer, many second-graders will assure you, is 60. (They know your question involves numbers, but they don't quite get the concept.)

Clinical psychologists are a bit like that. Most of us were not drawn to the field by a deep and abiding passion for numbers. This puts some in a quagmire when they jump into forensic work, and courts order them to predict future events with a high degree of mathematical precision.

Illusion of certainty, certitude in illusions

Since passage of the Psychologist Full Employment Act,* I have observed a growing group-think among government sex offender evaluators in particular. At the annual meetings of the Association for the Treatment of Sexual Abusers (ATSA), crowds flock to hear self-appointed gurus give the latest in a series of ever-changing instructions about how to use their pet formulas, freely available online, that promise to take the guesswork out of risk assessment.

Psychologists who lack statistical sophistication are especially likely to swoon over fancy-sounding terms such as receiver operating characteristics (ROC) and to overlook the gaping flaws in current actuarial methodology. Instead of deriving from sound scientific principles such as random sampling, the tools are strung together from a motley collection of random data, much of it never published or subjected to peer review. As I have reported in the past, the Static-99 family of instruments are not very accurate, and tend to err in the direction of overestimating risk.

So, what about that bus driver?

Getting back to the bus driver: Let's say the second-graders happened to be right, and he really is 60 years old. What are the odds that he will be arrested for a sex crime, given that he recently served time for sexual assault? (I know, I know. What bus company would have hired him? But, play along with me here.)

If you asked a randomly selected passenger aboard the bus, the answer would be close to 100%.

But as you know, the public drastically overestimates sex offender recidivism rates. Let’s say that in reality, the average sex offender who scores in the low range on actuarial risk instruments has a 5% chance of sexual recidivism, while the average high-scoring offender has a 29% risk. Obviously, without knowing more about the bus driver, all you can say is that his risk of reoffense is somewhere between 5% and 29%.

But that too would be wrong. Because of his age, the bus driver's recidivism risk over the next eight years is more in the range of 2.7%.

Which is probably lower than the risk of a passenger getting trampled if you hollered out, “Eek! Sex offender!”

Good news: Age-stratified tables improve accuracy

The single most robust finding of two centuries of criminological research is that desistance from crime is near universal. As they age, criminals stop offending. This holds true across all eras, cultures, and offender groups. Sex offenders are not exempt from this pattern. As their libidos decline, they too settle down or burn out. Unfortunately, this “age invariance effect,” as it has been called, has trouble filtering down into the muddy waters of the sex-offender industry. (See my online review of the book Desistance in the Open Access Journal of Forensic Psychology for more discussion of this.)

When age is not properly taken into accounting in estimating risk, the risk for older offenders -- such as our bus driver -- is overestimated, while the risk of younger offenders is underestimated.

Now, a collaboration by scholars from the United States, New Zealand, and Australia reveals that the accuracy of sex offender risk prediction can be significantly improved by using age-stratified tables to calculate risk.

The researchers tapped into an electronic database of all sex offenders in New Zealand who were released from prison over a 15-year period. They combined the data on those 5,880 offenders with recidivism data on 3,425 offenders published by Static-99 developer Karl Hanson in 2006, to develop what they call a "Multisample Age-Stratified Table of Sexual Recidivism Rates" (MATS-1).

Using Bayes's Theorem, the researchers were able to calculate likelihood ratios for different levels of risk. (Bayes's Theorem speaks to the probability of an event, taking into account both the phenomenon's base rate and the accuracy of a test. Cognitive scientists regard the Bayesian method as the gold standard, often using it synonymously with rational reasoning.)

Overall, the recidivism base rate of their combined international sample was 9% over a 10-year period, which is consistent with other reported research. Dividing offenders into three levels of risk based on their scores on actuarial risk instruments, the researchers found that those with low risk scores had an average 5% risk of reoffense within eight years, as compared with 12% for medium-risk offenders and 29% for offenders with high scores. By dividing sex offenders into various age groups, they were able to come up with more precise estimates of risk (see below table).



Evaluators should use this type of age-stratified procedure when giving estimates of recidivism risk, particularly for older offenders, the researchers advise. Estimating an offender's probability of recidivism based on the observed proportion of recidivists in a population is more accurate than relying on a set of untested assumptions. It is also much simpler and easier to explain to a trier of fact.

I highly recommend the article, published in the current issue of Sexual Abuse, which goes into a great deal of detail about the method and its superior stability and accuracy. The authors are Richard Wollert of Washington State University and the Mental Health Law and Policy Insitute at Simon Fraser University in Canada, Elliot Cramer, a statistician and professor emeritus from the University of North Carolina-Chapel Hill, Jacqueline Waggoner of the University of Portland, Alex Skelton of the New Zealand Department of Corrections, and James Vess of Deakin University in Australia. Request reprints from the first author (HERE).

Related blog posts:

• Delusional campaign for a world without risk (April 3, 2010)
• Static redux: Sandgropers jumping off rickety ship (Sept. 23, 2010)
• SVP industry sneak peek: Problems in Actuaryland (Oct. 4, 2009)
• New study: Do popular actuarials work? (April 20, 2010)
• Study: Actuarials fail to predict sexually violent recidivism (March 5, 2010)

For a good introduction to Bayesian reasoning, see Eliezer Yudkowsky's tutorial, "An Intuitive Explanation of Bayes' Theorem."

*The Psychologist Full Employment Act is the label conferred on the Sexually Violent Predator (SVP) laws by a leading psychology-law scholar in a recent plenary address.