United States

Biased Risk Assessment in COMPAS Recidivism Tool

The COMPAS software (Correctional Offender Management Profiling for Alternative Sanctions) is a decision support tool used by U.S. courts to assess the likelihood of a defendant would re-offend. The COMPAS software uses an algorithm to assess potential general recidivism risk general, potential violent recidivism, and risk for pretrial misconduct. COMPAS did not include race in calculating its risk scores. However, in 2016, ProPublica journalists investigated COMPAS and found that the system was far more likely to say black defendants were at risk of reoffending than their white counterparts. COMPAS software misclassified almost twice as many black defendants (45%) as higher risk compared to white defendants (23%), mistakenly labelled more white defendants as low risk, who then went on to reoffend – 48% white defendants compared to 28% black defendants and classified black defendants as higher risk when all other variables (such as prior crimes, age, and gender) were controlled – 77% more likely than white defendants. Some researchers later found that COMPAS algorithm is no better at predicting recidivism than random people, which raises question how reasonable it is to use these algorithms at all.

OUTCOME

The disclosure of COMPAS bias by ProPublica in 2016 led to multiple significant outcomes across public discourse, research, policy, and legal discussions, but few direct policy changes. The U.S. judiciary and policymakers started exploring AI ethics guidelines, but COMPAS continues to be used in many jurisdictions, with only some local or state-level reviews or replacements. There was no national moratorium or legal ban on using COMPAS or similar tools. The company behind COMPAS defended the tool, arguing that it was equally accurate across racial groups, but transparency remains an issue since COMPAS is still a black-box model.

REFERENCES

ProPublica “Machine Bias” series, 2016
MIT/Rudin transparency analysis 2018 (GitHub)
Wikipedia “COMPAS (software)” (Wikipedia)
Can You Make AI Fairer than a Judge? Play Our Courtroom Algorithm Game (MIT Technology Review)
Injustice Ex Machina: Predictive Algorithms in Criminal Sentencing (UCLA Law Review)
Machine Bias (ProPublica)
A Popular Algorithm Is No Better at Predicting Crimes Than Random People (The Atlantic)

Type of Bias – Technical Point

Algorithmic Bias

The COMPAS model exhibits unequal false positive and negative rates across races, incorrectly flagging 45% of Black defendants (who did not re-offend) as high-risk, versus 23% of white defendants (ProPublica).

Measurement / Proxy Bias

Even without race as input, proxies such as criminal history and age correlate with race, embedding bias indirectly.

Transparency / Model Bias

Proprietary, non-transparent nature prevents validation or error checking.

Type of Bias – Social Sciences Point

Racial Bias

Black defendants are nearly twice as likely to be misclassified as high risk; white defendants more likely to be misclassified as low-risk and reoffend.

Confirmation Bias

Courts and judges may over-trust scores labeled as “risk” without questioning, leading to automation bias (SpringerLink).

How many people were affected

Magnitude

ProPublica evaluated COMPAS data on 18,610 cases from Broward County, Florida, who were scored between 2013–2014 (source), and ~45% of non-reoffending Black defendants misclassified high-risk. However over 1 million individuals have been assessed using COMPAS’s recidivism risk scale since it was developed in 1998 (source). Tool remains in use across multiple U.S. jurisdictions.

REFERENCES

ProPublica “Machine Bias” series, 2016
MIT/Rudin transparency analysis 2018 (GitHub)
Wikipedia “COMPAS (software)” (Wikipedia)
Can You Make AI Fairer than a Judge? Play Our Courtroom Algorithm Game (MIT Technology Review)
Injustice Ex Machina: Predictive Algorithms in Criminal Sentencing (UCLA Law Review)
Machine Bias (ProPublica)
A Popular Algorithm Is No Better at Predicting Crimes Than Random People (The Atlantic)