Are Two Risk Assessments Better Than One?
Part 1: Sex Offender Risk Assessments

      In many forensic institutions, comprehensive assessments are provided by a multidisciplinary team to evaluate various aspects of a patient's functioning. Efforts are made to use standard evidence-based assessments. Detailed psychosocial histories permit in-depth assessment of future violence risk. As more risk assessment tools become available, it seems reasonable to combine them somehow, but a study published in the journal Psychological Assessment in June suggests that trying to use more than one assessment to do the same job is not a good idea.

      In his article, "Is more better? Combining actuarial risk scales to predict recidivism among adult sex offenders," Michael Seto from the Centre for Addiction and Mental Health (CAMH) compared four actuarial tools used to predict recidivism by sex offenders: Rapid Risk Assessment for Sexual Offense Recidivism (RRASOR), Static-99, Sex Offender Risk Appraisal Guide (SORAG), and the Violence Risk Appraisal Guide (VRAG). Each one of these tools is a good predictor. Seto first tested each tool on its own and found the best tool for his sample of sex offenders. Then he tried three rules for adding the other tools.

      The first rule was called, "Believe-the-negative." If the offender's score on at least one of the risk assessment tools was lower than a certain cut-off, Seto predicted that the offender would not reoffend. In the second rule, "Believe-the-positive," Seto predicted the offender would reoffend if his score on at least one of the tools was higher than the cut-off. For the third rule, the results of each tool were averaged, so that a relatively high score on one assessment was cancelled out by a relatively low score on another assessment. The prediction rules were tested across a range of possible cut-offs in a sample where recidivism had been assessed.

      None of the combinations rules significantly improved the accuracy of the best single prediction. In most cases, using more than one risk assessment made the prediction slightly worse.

      How could including more than one good assessment tool make the resulting assessment worse? Basically, the best tool already works well, so merging it with the second best tool has no value. "Believing the negative" means that all the tools have to be equally good at identifying recidivists; some recidivists will be missed because the weaker tools are more likely to give a low score to a recidivist in error (false negative).

      "Believing the positive" means that the tools have to be equally good at identifying non-recidivists, or the weaker tools will be more likely to give a high score to a nonrecidivist in error (false positive). Researchers tell how good a risk assessment is by calculating accuracy statistics that measure how well a tool identifies both recidivists and non-recidivists. Seto's research lends further support to using this evidencebased approach to selecting an assessment tool to use.

      Seto also used a variety of statistical procedures to search for a way to combine the tools successfully. No improvements were found using these procedures. Because these procedures are more rigorous and evidence-based than the three rules, Seto concluded that there is little chance that any benefits will be found from combining risk assessment tools.

      So what is the best tool for predicting recidivism among sex offenders? Seto actually tested two outcomes. For one test, he predicted sexual offenses involving physical contact that are easily identified as sexual from police charges. The best predictor for this outcome was the RRASOR, which was designed to predict sexual charges, by Karl Hanson of Public Safety and Emergency Preparedness Canada. For the second test, Seto predicted serious recidivism: essentially any sexual charges or nonsexual violent offenses. He did this because recent studies show that many nonsexual charges in sex offenders' police records are actually sexually motivated. (For example, sexual murder just appears as a homicide on a police record.) The best predictor of this serious recidivism among sex offenders was the SORAG, which was designed to predict this outcome, by MHCP researchers in the 1990’s. The SORAG is similar to the VRAG, but designed to predict the behaviour of sex offenders specifically.

      Seto's conclusions apply to sex offender recidivism specifically. They concern tools that are all designed to assess the same population. The findings also appear at first glance to contradict an earlier study in which combining the components of the HCR-20 tool improved prediction of physical violence. In Part 2 of this article, I will discuss these limits and describe MHCP researchers' attempts to improve prediction of domestic violence by combining risk assessment tools.