December 1, 2010

Age tables improve sex offender risk estimates

First, how old is the bus driver?

If 30 people are riding on a bus, and 30 more people board the bus, how old is the bus driver?

The answer, many second-graders will assure you, is 60. (They know your question involves numbers, but they don't quite get the concept.)

Clinical psychologists are a bit like that. Most of us were not drawn to the field by a deep and abiding passion for numbers. This puts some in a quagmire when they jump into forensic work, and courts order them to predict future events with a high degree of mathematical precision.

Illusion of certainty, certitude in illusions

Since passage of the Psychologist Full Employment Act,* I have observed a growing group-think among government sex offender evaluators in particular. At the annual meetings of the Association for the Treatment of Sexual Abusers (ATSA), crowds flock to hear self-appointed gurus give the latest in a series of ever-changing instructions about how to use their pet formulas, freely available online, that promise to take the guesswork out of risk assessment.

Psychologists who lack statistical sophistication are especially likely to swoon over fancy-sounding terms such as receiver operating characteristics (ROC) and to overlook the gaping flaws in current actuarial methodology. Instead of deriving from sound scientific principles such as random sampling, the tools are strung together from a motley collection of random data, much of it never published or subjected to peer review. As I have reported in the past, the Static-99 family of instruments are not very accurate, and tend to err in the direction of overestimating risk.

So, what about that bus driver?

Getting back to the bus driver: Let's say the second-graders happened to be right, and he really is 60 years old. What are the odds that he will be arrested for a sex crime, given that he recently served time for sexual assault? (I know, I know. What bus company would have hired him? But, play along with me here.)

If you asked a randomly selected passenger aboard the bus, the answer would be close to 100%.

But as you know, the public drastically overestimates sex offender recidivism rates. Let’s say that in reality, the average sex offender who scores in the low range on actuarial risk instruments has a 5% chance of sexual recidivism, while the average high-scoring offender has a 29% risk. Obviously, without knowing more about the bus driver, all you can say is that his risk of reoffense is somewhere between 5% and 29%.

But that too would be wrong. Because of his age, the bus driver's recidivism risk over the next eight years is more in the range of 2.7%.

Which is probably lower than the risk of a passenger getting trampled if you hollered out, “Eek! Sex offender!”

Good news: Age-stratified tables improve accuracy

The single most robust finding of two centuries of criminological research is that desistance from crime is near universal. As they age, criminals stop offending. This holds true across all eras, cultures, and offender groups. Sex offenders are not exempt from this pattern. As their libidos decline, they too settle down or burn out. Unfortunately, this “age invariance effect,” as it has been called, has trouble filtering down into the muddy waters of the sex-offender industry. (See my online review of the book Desistance in the Open Access Journal of Forensic Psychology for more discussion of this.)

When age is not properly taken into accounting in estimating risk, the risk for older offenders -- such as our bus driver -- is overestimated, while the risk of younger offenders is underestimated.

Now, a collaboration by scholars from the United States, New Zealand, and Australia reveals that the accuracy of sex offender risk prediction can be significantly improved by using age-stratified tables to calculate risk.

The researchers tapped into an electronic database of all sex offenders in New Zealand who were released from prison over a 15-year period. They combined the data on those 5,880 offenders with recidivism data on 3,425 offenders published by Static-99 developer Karl Hanson in 2006, to develop what they call a "Multisample Age-Stratified Table of Sexual Recidivism Rates" (MATS-1).

Using Bayes's Theorem, the researchers were able to calculate likelihood ratios for different levels of risk. (Bayes's Theorem speaks to the probability of an event, taking into account both the phenomenon's base rate and the accuracy of a test. Cognitive scientists regard the Bayesian method as the gold standard, often using it synonymously with rational reasoning.)

Overall, the recidivism base rate of their combined international sample was 9% over a 10-year period, which is consistent with other reported research. Dividing offenders into three levels of risk based on their scores on actuarial risk instruments, the researchers found that those with low risk scores had an average 5% risk of reoffense within eight years, as compared with 12% for medium-risk offenders and 29% for offenders with high scores. By dividing sex offenders into various age groups, they were able to come up with more precise estimates of risk (see below table).

Evaluators should use this type of age-stratified procedure when giving estimates of recidivism risk, particularly for older offenders, the researchers advise. Estimating an offender's probability of recidivism based on the observed proportion of recidivists in a population is more accurate than relying on a set of untested assumptions. It is also much simpler and easier to explain to a trier of fact.

I highly recommend the article, published in the current issue of Sexual Abuse, which goes into a great deal of detail about the method and its superior stability and accuracy. The authors are Richard Wollert of Washington State University and the Mental Health Law and Policy Insitute at Simon Fraser University in Canada, Elliot Cramer, a statistician and professor emeritus from the University of North Carolina-Chapel Hill, Jacqueline Waggoner of the University of Portland, Alex Skelton of the New Zealand Department of Corrections, and James Vess of Deakin University in Australia. Request reprints from the first author (HERE).

Related blog posts:
For a good introduction to Bayesian reasoning, see Eliezer Yudkowsky's tutorial, "An Intuitive Explanation of Bayes' Theorem."

*The Psychologist Full Employment Act is the label conferred on the Sexually Violent Predator (SVP) laws by a leading psychology-law scholar in a recent plenary address.


Neil B Fisher said...

My question is this: Our governments (states and federal)won't even admit or allow to the general population that the major majority of the information that the general population were made to belive (regarding sex offenders)as being the truth is infact wrong,false,myths.So what is the chances that these new exams (being stated as being better)are even going to be allowed to be shown as being a better exam(what are the chances that our government is going to even allow for anything that will show the ability to lower the number of dangerous sexual offenders as per to the ones that are not. As I have stated untill there is some major ground breaking damage repair then damage control as to get the attitude of the general population to change as to get far better law,rules and regulations into law as relating to sex offenders.
There is to much power and funding and a everlasting positive cash flow in the way the sex offender system is set up now(the sex offender industry in general).Also the way things are set up now it allows for a DA to find a sex offender guilty in a court of law and I can't see any government making things harder to win overall and the way it is set up it allows for our governments to make it appear that it is not soft on crime and sex offenders has been made out to be the worst crime over all. I suspect that the best bet is to allow for the makers of these newer exams to somehow put pressure upon our governments to allow for a peoples postive attitude. Anyway you look at it there has to be some kind of damage repair made due to the amount of damage that the false,wrong and myths has made relating to sex offenders.There is also just so much information being brought foward regarding the sexual violent predator that the issue is being overly clouded to the point that somewhere in that awray of information the correct awnser is probably in there somewhere. I myself have not even found anyone with some courage to talk a class action to force our govenments to tell the population the true facts (its just not politicaly currectat this time. I also truly believe that the California Sex Offender Management Board (CASOMB) Being it was the state of California that brought the tracking that led to lifetime registration in the first place and that the majority of other governments use California laws as a template to their own sex offender laws the majority of information should be brought to the CASOMB attention being it is this board that will be making suggestions to the heads of state relating to changes in the sex offender laws but will California allow for the suggestions given by the board? time will tell

Karen Franklin, Ph.D. said...

You are commenting on an older post. More people will see it if you copy and paste it as a comment to today's post, regarding the new MnSOST-3 instrument.