What we don’t know about risk assessment in the justice system

In the criminal justice system, data has become a hot topic, and often the driving force behind decisions related to law enforcement activities, pretrial release on bail, sentencing decisions, and release on parole. It is commonly believed that predictions related to recidivism that are based on algorithms and risk assessment measures will help reduce mass incarceration and the harsh collateral consequences that disproportionately impact minorities and the poor. However, this belief in the accuracy and fairness of numbers-based decision-making is based more on faith than it is on facts.

Risk assessment related to criminal recidivism is based on a variety of factors statistically correlated with criminal behavior. Some factors that are included in risk assessment tools and algorithms include age, gender, criminal history, socioeconomic status, mental illness, substance abuse, and education level, among others.

By their very nature, risk assessment tools apply group statistics to individuals. Decisions are then made about the individual (for example, whether he or she should be released on bail, or sentenced to jail/prison or probation) based on his or her shared characteristics with groups of people who have committed a new offense.

One of the primary arguments in favor of using risk assessment tools in criminal justice decisions (for example, sentencing) is that they will counteract various forms of explicit or implicit racial bias. The disparity in sentencing outcomes between whites and other minority groups is well-documented in the U.S. criminal justice system.

Much of the disparity is thought to be related to biased assumptions on the part of judges about individuals that are based on stereotypes. The question is: do risk assessment tools and algorithms overcome unfair stereotypes, or do they provide a way to make stereotypes appear objective?

Questions about the fairness and racial neutrality of risk assessment are now being raised with increasing frequency. The fact is that there is very little research that examines how risk assessment tools and algorithms impact decision-making in the justice system.

In other words – do judges actually take risk assessments into account in their decisions? Some studies indicate that risk assessment matters little to judges. Furthermore, there are serious questions about the potential for risk tools to overclassify minorities as high risk or underclassify whites as low risk that require additional study.

Finally, a question seldom considered in discussions about using risk assessment tools is how risk assessment actually impacts recidivism. A recent article in Law and Human Behavior suggests that professionals frequently fail to follow the recommendations of risk assessment tools. In addition, interventions designed to address factors that might reduce recidivism are often non-existent, or only available to a very small percentage of individuals who need them.

The authors of that study, Viljoen, Cochrane, and Johnson (2018) conclude that the use of risk assessment tools does not automatically translate into effective treatment or risk management. Identifying offender needs without addressing those needs may simply result in individuals assessed as “high risk” being subject to additional punishment.

Conclusions about the impact of risk assessment on criminal justice decision-making are premature, as are conclusions about whether risk assessment tools and algorithms have a disproportionately negative impact on minorities and the poor. In addition, the assessment of risk does not guarantee interventions matched to risk reduction, and can mean more punishment for some.

Very little research has been done on these questions of primary importance, suggesting that widespread implementation of these tools is ill-advised without considering the real consequences rather than what we hope will happen.