James P. Scanlan, Attorney at Law

Recidivism Illustration

(Jan. 4, 2021; cor. March 4, 2023)

This page is one of many pages on this website using published data to illustrate the pattern whereby when two groups differ in their susceptibility to an outcome, the rarer an outcome the greater tends to be the relative difference between rates at which the groups experience it and the smaller tends to be the relative difference between rates at which the groups avoid it.  The page is usefully examined with a recognition that virtually all social scientists and all government agencies enforcing civil rights laws believe that reducing adverse criminal justice, school discipline, lending, and other outcomes will tend to reduce (a) relative racial differences in rates of experiencing the outcomes (a measure commonly represented in terms of the ratio of the black rate to the white rate) and (b) the proportion blacks make up of persons experiencing the outcomes.  In fact, however, in accordance with the pattern described in the first sentence, generally reducing such outcomes tends to increase (a) and (b) for the outcomes.

That is, reducing any outcome tends to increase relative differences in rates of experiencing the outcomes (while reducing relative differences in rates of avoiding the outcome) and increase the proportion the more susceptible group makes up of persons experiencing the outcome (while also increasing the proportion the group make up of persons avoiding the outcome).   This page uses data from a May 23, 2016 ProPublica article titled “How We Analyzed the COMPASS Recidivism Algorithm” to show that raising the level of recidivism risk to call for incarceration of a criminal defendant – something that could also be characterized as relaxing the standards for release of the defendant – would tend to increase relative racial differences in rates at which defendants are incarcerated while reducing relative racial differences in rates at which defendants avoid incarceration.  Correspondingly, such action will tend to increase the proportion blacks make up of defendants who are incarcerated (thus increasing all measures of difference between the proportion blacks make up of defendants and the proportion they make up of incarcerated defendants) and increase the proportion blacks make up of the defendants who are released (thus reducing all measures of difference between the proportion blacks make up of defendants and the proportion they make up of defendants who are released).

The subject of the ProPublica article is actually the perceived unfairness in the employment of algorithms for predicting recidivism.  That perceived unfairness lies in the fact that, even though an algorithm underpredicts recidivism of blacks and overpredicts recidivism of whites, it results in a situation where, among persons who do not recidivate, blacks are more likely than whites to have been identified as likely to recidivate (so called false positives) and, among persons who do recidivate, whites are more likely than blacks to have been identified as unlikely to recidivate (so-called false negatives).   Statements like “black defendants were far more likely than white defendants to be incorrectly judged to be at a higher risk of recidivism, while white defendants were more likely than black defendants to be incorrectly flagged as low risk,” as in the ProPublica article, must be interpreted with that understanding.  That is, among all defendants, black defendants were in fact less likely to be incorrectly identified as a person who will recidivate than a white defendant and more likely than whites to be incorrectly identified as unlikely to recidivate.  It is only among defendants who did not recidivate that blacks were more likely than whites to have been incorrectly identified as highly likely to recidivate; and only among defendants who did recidivate that whites were more likely than blacks to have been incorrectly identified as highly unlikely to recidivate.  

This is the same issue addressed with regard to employment tests in Ability Testing: Uses, Consequences and Controversies, Part I, National Academies Press (1982) (Committee on Ability Testing, Assembly of Behavioral and Social Sciences, National Research Council, Alexandra K. Wigdor & Wendell R. Garner (eds.)) and various others places in the 1980s and early 1990s.  In that situation, however, persons considered false negatives experienced the favorable outcome (i.e., performing well on the job even though the test predicted they would not, or, more specifically, predicted a lower chance of performing well than was desired for selection) while persons considered false positives experienced the adverse outcome( i.e., performing poorly on the job even though the test predicted they would perform well (or, more specifically, predicted a chance of performing well that was higher than was desired as a minimum chance of successful performance in order to be selected for the job).[i]  For clarity, the favorable and adverse outcomes discussed immediately above are successful and unsuccessful job performance, not selection and non-selection.  

I will eventually create a page regarding the failure to understand how various factors affect measures of differences in the rates at which different groups experience false positives or false negatives.  This page, however, is largely limited to showing how data in the ProPublica article illustrate the patterns described in the first paragraph and may be compared to the following pages in this regard:  Obesity Illustrations, Framingham Illustrations, NHANES Illustrations,  Life Tables Illustrations, Income Illustrations, Credit Score Illustrations, Numeracy Illustration, Sears Case Illustration, Literacy Illustration, California Prison Disparities.

Tables 1 and 2 of this page use data from the first bar chart in the ProPublica article to illustrate that the more liberal a release policy, the greater will tend to be relative racial differences incarceration rates (while the smaller tend to be relative racial differences in release relates) and the larger will tend to be the proportion blacks make up of both persons incarcerated and persons released.  The misunderstanding of policies on measures of racial disparity in criminal justice outcomes are discussed the “Usual, But Wholly Misunderstood, Effects of Policies on Measures of Racial Disparity Now Being Seen in Ferguson and the UK and Soon to Be Seen in Baltimore,” Federalist Society Blog (Dec. 4, 2019); “United States Exports Its Most Profound Ignorance About Racial Disparities to the United Kingdom,” Federalist Society Blog (Nov. 2, 2017), “The Pernicious Misunderstanding of Effects or Policies on Racial Differences in Criminal Justice Outcomes,” Federalist Society Blog (Oct. 12, 2017), “Racial Impact Statement Laws in New Jersey and Elsewhere,” Federalist Society Blog (Mar. 20, 2017), and long ago in “Mired in Numbers,” Legal Times (Oct. 12, 1996).[ii]   See especially the Appendix to the first item (titled “Innumeracy in the Disparities Industry”) regarding the way the entities like the Georgetown University Center for Juvenile Justice reform charge program participants substantial sums while leading them falsely to believe that things like diversion programs will tend to reduce, rather than increase, relative racial differences in rates of proceeding further into the criminal justice system.  The underlying principles are discussed in the context of the broader problems in the analyses of demographic differences due to the failure to understand the ways standard measures of differences between outcome rates tend to be affected by the prevalence of an outcome are discussed in “Race and Mortality Revisited,” Society (July/Aug. 2014), Comments for the Commission on Evidence-Based Policymaking (Nov. 14, 2016), “The Mismeasure of Discrimination,” Faculty Workshop, University of Kansas School of Law (Sept. 20, 2013), Letter to Harvard University  (Oct. 9, 2012).  

Table 1 below is based on the bar chart in the ProPublica article showing the numbers of blacks and whites in the sample falling into ten recidivism score ranges.  The numbers underlying the table are based on estimation (using the ruler function in a PDF program) of the numbers reflected by each bar.  Thus, certain estimations may be slightly incorrect, as reflected by the fact that estimated figures in each range sum to 3215.03 black defendants and 2128.32 white defendants whereas the actual numbers shown in the article are 3175 black defendants and 2103 white defendants.[iii]  But it is inconceivable that the precisely correct figures would show a general departure from the pattern in the tables and unlikely that it would show a departure from the pattern in any single row of data.

The first column shows the cut point for release.  The first row reflects a stringent standard for release whereby only those with risk levels in the first decile (that is, below the second decile) are released, and the bottom row reflects a lenient standard where all persons with risk levels below the tenth decile are released.[iv]  (As suggested earlier, lenient standards for release could also be characterized as stringent standards for incarceration.).  The next four columns show the numbers and proportions of black and white defendants that would be released using the cut point indicated in the first column.  The fourth and third last columns then show the relative differences in rates of being held and relative differences in rates of being released (with the first case in terms of the ratio of the black rate of being incarcerated to the white rate of being incarcerated and the latter cast in terms of the ratio of the white rate of being released to the black rate of being released[v]).  In accordance with the patterns described at the outset of this page, the two columns show that as the standards for release are relaxed, thereby generally reducing rates of being incarcerated and generally increasing rates of being released, relative racial differences in rates of being incarcerated increase while relative differences in rates of being released decrease.  The final two columns show that, also in accord with the patterns described at the outset, as the standards for release are made more lenient, the proportion blacks make up of persons held and the proportion blacks make up of persons released both increase. 

Table 1:  Effects of modifying recidivism risk levels for release on relative racial differences in rates of being held and rates of being released and proportions black defendants make up of persons held and persons released – all offenses.

Table 2 presents the same information as Table 1 but using the ProPublica information on risk levels for violent offenses as presented in the second set of bar charts in the article.  Again, the numbers of blacks and whites in each risk level decile the number are somewhat inexact, as reflected by the fact that the article states that the total numbers of black and whites in the analyses were 1918 blacks and 1459, while the estimated numbers for each risk level sum to 1880.49 for blacks and 1419.51 for whites. 

Table 2:  Effects of modifying recidivism risk levels for release on relative racial differences in rates of being held and rates of being released and proportions black defendants make up of persons held and persons released – violent offenses.

Like Table 1, Table 2 shows a general pattern whereby the more lenient the standard for release, the greater is the relative difference in rates of incarceration and the smaller is the relative difference in rates of release.  Also, as with Table 1, the more lenient the standard for release the larger is the proportion blacks make up both of defendants incarcerated and defendants released.  There is, however, a departure from the general pattern in the penultimate row, which, compared with the prior row, shows a decrease in the relative difference in rates of being incarcerated and a decrease in the proportion blacks make up of incarcerated persons.  Such departure is a function of an irregularity in the distributions at a point where there are comparatively few observations, especially for whites, or (probably less likely) inexactness in my estimations of numbers reflected by each bar in the bar charts.

One will note that at each cut point the relative difference in rates of being incarcerated are larger, while relative differences in rates of being released are smaller for violent offenses than for all offenses.  Such a pattern is to be expected given that risks of violent recidivism are much lower than risk of any recidivism (which is not to say that the relationship of the black and white risk distributions for violent offenses is the same as for all offenses[vi]). 

Other data in the ProPublica article also reflect aspects of the pattern described at the outset.  One corollary to the pattern is that a factor that affects outcome rates for two groups with different baseline rates for the outcome will tend to cause a larger proportionate change in the outcome rate for the group with the lower baseline rate for the outcome while causing a larger proportionate change in the opposite outcome rate for other group.  See "Race and Mortality Revisited" (at 340-341) and the Subgroup Effects and Subgroup Effects – Nonclinical pages.  Given that whites have lower recidivism rates among persons designated low risk, having a higher risk rate will tend to cause larger proportionate increases in recidivism for whites than blacks while causing larger proportionate decrease in rates of not recidivating for blacks than whites.  The data in the ProPublica two-by-two tables for general recidivism and for violent recidivism (which treat all risks ranges of five or higher as high risk) show such a pattern.  That is, for recidivism generally the ratio of the recidivism rate for the high risk group (decile 5 or above) to the low risk group (decile 4 or below) was 2.05 for whites and 1.80 for blacks, while the ratio of the rate of no recidivism for the low risk group to the high risk group was 1.74 for whites and 1.76 for blacks; for violent recidivism the ratio of the recidivism rate for the high risk group to the rate for the low risk group was 2.36 for whites and 1.27 for blacks, while the ratio of the rate of no recidivism for the low risk group to the rate for the high risk group was 1.12 for whites and 1.14 for blacks

As reflected in the references, however, this pattern is entirely misunderstood and observers mistakenly attribute significance to differing relative effects on whichever outcome they happen to be examining.  Comparing deciles 8 to 10 with deciles 1 to 3, the ProPublica article notes that ratios of recidivism rates of the higher risk group to the lower risk group are 3.61 for whites and 2.99 for blacks.  Comparing deciles 5 to 7 with deciles 1 to 4, the article notes that ratios of recidivism rates of the higher risk group to the lower risk group are 2.23 for whites and 1.95 for blacks.  Such patterns are in accord with the pattern just described and the underlying data would almost certainly show that being in the higher risk category reduces the rates of avoiding recidivism proportionately more for blacks than whites.

The ProPublica article, however, reads the larger relative increase in recidivism for whites than blacks as if it were significant, noting that it shows that the score is performing differently for the two racial groups.  This statement reflects the illogical expectation that in the normal course a factor that affects outcome rates will cause proportionate changes in the outcome rates for group with different baseline rates.  The expectation is illogical because it is mathematically impossible for a factor to cause equal proportionate changes in different baseline rates for an outcome while causing equal proportionate changes in the opposite outcome.  See references in the last paragraph and the  Illogical Premises page. 

The data in the two-by-two tables for general recidivism and for violent recidivism also show that, both among persons designated high risk and persons designated low risk, blacks are more likely to recidivate than whites.  The bar charts illustrate why that it is to be expected.  That is, within each broad risk category, blacks have higher risk scores than whites.