James P. Scanlan, Attorney at Law

Home Page

Curriculum Vitae

Publications

Published Articles

Conference Presentations

Working Papers

page1

Journal Comments

Truth in Justice Articles

Measurement Letters

Measuring Health Disp

Outline and Guide to MHD

Summary to MHD

Solutions

page3

Solutions Database

Irreducible Minimums

Pay for Performance

Between Group Variance

Concentration Index

Gini Coefficient

Reporting Heterogeneity

Cohort Considerations

Relative v Absolute Diff

Whitehall Studies

AHRQ's Vanderbilt Report

NHDR Measurement

NHDR Technical Issues

MHD A Articles

MHD B Conf Presentations

MHD D Journal Comments

Consensus/Non-Consensus

Spurious Contradictions

Institutional Corresp

page2

Scanlan's Rule

Outline and Guide to SR

Summary to SR

Bibliography

Semantic Issues

Employment Tests

Case Study

Case Study Answers

Case Study II

Subgroup Effects

Subgroup Effects NC

Illogical Premises

Illogical Premises II

Inevitable Interaction

Interactions by Age

Literacy Illustration

RERI

Feminization of Poverty S

Explanatory Theories

Mortality and Survival

Truncation Issues

Collected Illustrations

Income Illustrations

Framingham Illustrations

Life Table Illustrations

NHANES Illustrations

Mort/Surv Illustration

Credit Score Illustration

Intermediate Outcomes

Representational Disp

Statistical Signif SR

Comparing Averages

Meta-Analysis

Case Control Studies

Criminal Record Effects

Sears Case Illustration

Numeracy Illustration

Obesity Illusration

LIHTC Approval Disparitie

Recidivism Illustration

Consensus

Algorithm Fairness

Mortality and Survival 2

Mort/Survival Update

Measures of Association

Immunization Disparities

Race Health Initiative

Educational Disparities

Disparities by Subject

CUNY ISLG Eq Indicators

Harvard CRP NCLB Study

New York Proficiency Disp

Education Trust GC Study

Education Trust HA Study

AE Casey Profic Study

McKinsey Achiev Gap Study

California RICA

Nuclear Deterrence

Employment Discrimination

Job Segregation

Measuring Hiring Discr

Disparate Impact

Four-Fifths Rule

Less Discr Alt - Proc

Less Discr Altl - Subs

Fisher v. Transco Serv

Jones v. City of Boston

Bottom Line Issue

Lending Disparities

Inc & Cred Score Example

Disparities - High Income

Underadjustment Issues

Absolute Differences - L

Lathern v. NationsBank

US v. Countrywide

US v. Wells Fargo

Partial Picture Issues

Foreclosure Disparities

File Comparison Issues

FHA/VA Steering Study

CAP TARP Study

Disparities by Sector

Holder/Perez Letter

Federal Reserve Letter

Discipline Disparities

COPAA v. DeVos

Kerri K. V. California

Truancy Illustration

Disparate Treatment

Relative Absolute Diff

Offense Type Issues

Los Angeles SWPBS

Oakland Disparities

Richmond Disparities

Nashville Disparities

California Disparities

Denver Disparities

Colorado Disparities

Nor Carolina Disparitie

Aurora Disparities

Allegheny County Disp

Evansville Disparities

Maryland Disparities

St. Paul Disparities

Seattle Disparities

Minneapolis Disparities

Oregon Disparities

Beaverton Disparities

Montgomery County Disp

Henrico County Disparitie

Florida Disparities

Connecticut Disparities

Portland Disparities

Minnesota Disparities

Massachusetts Disparities

Rhode Island Disparities

South Bend Disparities

Utah Disparities

Loudoun Cty Disparities

Kern County Disparities

Milwaukee Disparities

Urbana Disparities

Illinois Disparities

Virginia Disparities

Behavior

Suburban Disparities

Preschool Disparities

Restraint Disparities

Disabilities - PL 108-446

Keep Kids in School Act

Gender Disparities

Ferguson Arrest Disp

NEPC Colorado Study

NEPC National Study

California Prison Pop

APA Zero Tolerance Study

Flawed Inferences - Disc

Oakland Agreement

DOE Equity Report

IDEA Data Center Guide

Duncan/Ali Letter

Crim Justice Disparities

U.S. Customs Search Disp

Deescalation Training

Career Criminal Study

Implicit Bias Training

Drawing Inferences

Diversion Programs

Minneapolis PD Investig

Offense Type Issues CJD

Innumerate Decree Monitor

Massachusetts CJ Disparit

Feminization of Poverty

Affirmative Action

Affirm Action for Women

Other Affirm Action

Justice John Paul Stevens

Statistical Reasoning

The Sears Case

Sears Case Documents

The AT&T Consent Decree

Cross v. ASPI

Vignettes

Times Higher Issues

Gender Diff in DADT Term

Adjustment Issues

Percentage Points

Odds Ratios

Statistical Signif Vig

Journalists & Statistics

Multiplication Definition

Prosecutorial Misconduct

Outline and Guide

Misconduct Summary

B1 Agent Cain Testimony

B1a Bev Wilsh Diversion

B2 Bk Entry re Cain Call

B3 John Mitchell Count

B3a Obscuring Msg Slips

B3b Missing Barksdale Int

B4 Park Towers

B5 Dean 1997 Motion

B6 Demery Testimony

B7 Sankin Receipts

B7a Sankin HBS App

B8 DOJ Complicity

B9 Doc Manager Complaints

B9a Fabricated Gov Exh 25

B11a DC Bar Complaint

Letters (Misconduct)

Links Page

Misconduct Profiles

Arlin M. Adams

Jo Ann Harris

Bruce C. Swartz

Swartz Addendum 2

Swartz Addendum 3

Swartz Addendum 4

Swartz Addendum 7

Robert E. O'Neill

O'Neill Addendum 7

Paula A. Sweeney

Robert J. Meyer

Lantos Hearings

Password Protected

OIC Doc Manager Material

DC Bar Materials

Temp Confidential

DV Issues

Indexes

Document Storage

Pre 1989

1989 - present

Presentations

Prosec Misc Docs

Prosec Misc Docs II

Profile PDFs

Misc Letters July 2008 on

Large Prosec Misc Docs

HUD Documents

Transcripts

Miscellaneous Documents

Unpublished Papers

Letters re MHD

Tables

MHD Comments

Figures

ASPI Documents

Web Page PDFs

Sears Documents

Pages Transfer


Recidivism Illustration

(Jan. 4, 2021; cor. March 4, 2023)

This page is one of many pages on this website using published data to illustrate the pattern whereby when two groups differ in their susceptibility to an outcome, the rarer an outcome the greater tends to be the relative difference between rates at which the groups experience it and the smaller tends to be the relative difference between rates at which the groups avoid it.  The page is usefully examined with a recognition that virtually all social scientists and all government agencies enforcing civil rights laws believe that reducing adverse criminal justice, school discipline, lending, and other outcomes will tend to reduce (a) relative racial differences in rates of experiencing the outcomes (a measure commonly represented in terms of the ratio of the black rate to the white rate) and (b) the proportion blacks make up of persons experiencing the outcomes.  In fact, however, in accordance with the pattern described in the first sentence, generally reducing such outcomes tends to increase (a) and (b) for the outcomes.

That is, reducing any outcome tends to increase relative differences in rates of experiencing the outcomes (while reducing relative differences in rates of avoiding the outcome) and increase the proportion the more susceptible group makes up of persons experiencing the outcome (while also increasing the proportion the group make up of persons avoiding the outcome).   This page uses data from a May 23, 2016 ProPublica article titled “How We Analyzed the COMPASS Recidivism Algorithm” to show that raising the level of recidivism risk to call for incarceration of a criminal defendant – something that could also be characterized as relaxing the standards for release of the defendant – would tend to increase relative racial differences in rates at which defendants are incarcerated while reducing relative racial differences in rates at which defendants avoid incarceration.  Correspondingly, such action will tend to increase the proportion blacks make up of defendants who are incarcerated (thus increasing all measures of difference between the proportion blacks make up of defendants and the proportion they make up of incarcerated defendants) and increase the proportion blacks make up of the defendants who are released (thus reducing all measures of difference between the proportion blacks make up of defendants and the proportion they make up of defendants who are released).

The subject of the ProPublica article is actually the perceived unfairness in the employment of algorithms for predicting recidivism.  That perceived unfairness lies in the fact that, even though an algorithm underpredicts recidivism of blacks and overpredicts recidivism of whites, it results in a situation where, among persons who do not recidivate, blacks are more likely than whites to have been identified as likely to recidivate (so called false positives) and, among persons who do recidivate, whites are more likely than blacks to have been identified as unlikely to recidivate (so-called false negatives).   Statements like “black defendants were far more likely than white defendants to be incorrectly judged to be at a higher risk of recidivism, while white defendants were more likely than black defendants to be incorrectly flagged as low risk,” as in the ProPublica article, must be interpreted with that understanding.  That is, among all defendants, black defendants were in fact less likely to be incorrectly identified as a person who will recidivate than a white defendant and more likely than whites to be incorrectly identified as unlikely to recidivate.  It is only among defendants who did not recidivate that blacks were more likely than whites to have been incorrectly identified as highly likely to recidivate; and only among defendants who did recidivate that whites were more likely than blacks to have been incorrectly identified as highly unlikely to recidivate.  

This is the same issue addressed with regard to employment tests in Ability Testing: Uses, Consequences and Controversies, Part I, National Academies Press (1982) (Committee on Ability Testing, Assembly of Behavioral and Social Sciences, National Research Council, Alexandra K. Wigdor & Wendell R. Garner (eds.)) and various others places in the 1980s and early 1990s.  In that situation, however, persons considered false negatives experienced the favorable outcome (i.e., performing well on the job even though the test predicted they would not, or, more specifically, predicted a lower chance of performing well than was desired for selection) while persons considered false positives experienced the adverse outcome( i.e., performing poorly on the job even though the test predicted they would perform well (or, more specifically, predicted a chance of performing well that was higher than was desired as a minimum chance of successful performance in order to be selected for the job).[i]  For clarity, the favorable and adverse outcomes discussed immediately above are successful and unsuccessful job performance, not selection and non-selection.  

I will eventually create a page regarding the failure to understand how various factors affect measures of differences in the rates at which different groups experience false positives or false negatives.  This page, however, is largely limited to showing how data in the ProPublica article illustrate the patterns described in the first paragraph and may be compared to the following pages in this regard:  Obesity Illustrations, Framingham Illustrations, NHANES Illustrations,  Life Tables Illustrations, Income Illustrations, Credit Score Illustrations, Numeracy Illustration, Sears Case Illustration, Literacy Illustration, California Prison Disparities.

Tables 1 and 2 of this page use data from the first bar chart in the ProPublica article to illustrate that the more liberal a release policy, the greater will tend to be relative racial differences incarceration rates (while the smaller tend to be relative racial differences in release relates) and the larger will tend to be the proportion blacks make up of both persons incarcerated and persons released.  The misunderstanding of policies on measures of racial disparity in criminal justice outcomes are discussed the “Usual, But Wholly Misunderstood, Effects of Policies on Measures of Racial Disparity Now Being Seen in Ferguson and the UK and Soon to Be Seen in Baltimore,” Federalist Society Blog (Dec. 4, 2019); “United States Exports Its Most Profound Ignorance About Racial Disparities to the United Kingdom,” Federalist Society Blog (Nov. 2, 2017), “The Pernicious Misunderstanding of Effects or Policies on Racial Differences in Criminal Justice Outcomes,” Federalist Society Blog (Oct. 12, 2017), “Racial Impact Statement Laws in New Jersey and Elsewhere,” Federalist Society Blog (Mar. 20, 2017), and long ago in “Mired in Numbers,” Legal Times (Oct. 12, 1996).[ii]   See especially the Appendix to the first item (titled “Innumeracy in the Disparities Industry”) regarding the way the entities like the Georgetown University Center for Juvenile Justice reform charge program participants substantial sums while leading them falsely to believe that things like diversion programs will tend to reduce, rather than increase, relative racial differences in rates of proceeding further into the criminal justice system.  The underlying principles are discussed in the context of the broader problems in the analyses of demographic differences due to the failure to understand the ways standard measures of differences between outcome rates tend to be affected by the prevalence of an outcome are discussed in “Race and Mortality Revisited,” Society (July/Aug. 2014), Comments for the Commission on Evidence-Based Policymaking (Nov. 14, 2016), “The Mismeasure of Discrimination,” Faculty Workshop, University of Kansas School of Law (Sept. 20, 2013), Letter to Harvard University  (Oct. 9, 2012).  

Table 1 below is based on the bar chart in the ProPublica article showing the numbers of blacks and whites in the sample falling into ten recidivism score ranges.  The numbers underlying the table are based on estimation (using the ruler function in a PDF program) of the numbers reflected by each bar.  Thus, certain estimations may be slightly incorrect, as reflected by the fact that estimated figures in each range sum to 3215.03 black defendants and 2128.32 white defendants whereas the actual numbers shown in the article are 3175 black defendants and 2103 white defendants.[iii]  But it is inconceivable that the precisely correct figures would show a general departure from the pattern in the tables and unlikely that it would show a departure from the pattern in any single row of data.

The first column shows the cut point for release.  The first row reflects a stringent standard for release whereby only those with risk levels in the first decile (that is, below the second decile) are released, and the bottom row reflects a lenient standard where all persons with risk levels below the tenth decile are released.[iv]  (As suggested earlier, lenient standards for release could also be characterized as stringent standards for incarceration.).  The next four columns show the numbers and proportions of black and white defendants that would be released using the cut point indicated in the first column.  The fourth and third last columns then show the relative differences in rates of being held and relative differences in rates of being released (with the first case in terms of the ratio of the black rate of being incarcerated to the white rate of being incarcerated and the latter cast in terms of the ratio of the white rate of being released to the black rate of being released[v]).  In accordance with the patterns described at the outset of this page, the two columns show that as the standards for release are relaxed, thereby generally reducing rates of being incarcerated and generally increasing rates of being released, relative racial differences in rates of being incarcerated increase while relative differences in rates of being released decrease.  The final two columns show that, also in accord with the patterns described at the outset, as the standards for release are made more lenient, the proportion blacks make up of persons held and the proportion blacks make up of persons released both increase. 

Table 1:  Effects of modifying recidivism risk levels for release on relative racial differences in rates of being held and rates of being released and proportions black defendants make up of persons held and persons released – all offenses.

Risk Level Below

Number

Blacks Below

Number Whites Below

Prop of Blacks Below

Prop of White Below

Ratio

Bl/Wh

Incar 

Ratio Wh/Bl Rel Rt

Bl Prop of Incar

Bl Prop of Rel

2

368.79

617.34

11.47%

29.01%

1.25

2.53

65.32%

37.40%

3

721.39

943.35

22.44%

44.32%

1.39

1.98

67.79%

43.33%

4

1026.59

1180.35

31.93%

55.46%

1.53

1.74

69.78%

46.52%

5

1369.94

1428.90

42.61%

67.14%

1.75

1.58

72.51%

48.95%

6

1699.42

1631.21

52.86%

76.64%

2.02

1.45

75.30%

51.02%

7

2017.34

1793.06

62.75%

84.25%

2.36

1.34

78.13%

52.94%

8

2364.16

1902.89

73.53%

89.41%

2.50

1.22

79.05%

55.41%

9

2670.52

2001.16

83.06%

94.02%

2.83

1.13

81.07%

57.16%

10

2988.44

2076.30

92.95%

97.56%

2.88

1.05

81.33%

59.00%

 

Table 2 presents the same information as Table 1 but using the ProPublica information on risk levels for violent offenses as presented in the second set of bar charts in the article.  Again, the numbers of blacks and whites in each risk level decile the number are somewhat inexact, as reflected by the fact that the article states that the total numbers of black and whites in the analyses were 1918 blacks and 1459, while the estimated numbers for each risk level sum to 1880.49 for blacks and 1419.51 for whites. 

Table 2:  Effects of modifying recidivism risk levels for release on relative racial differences in rates of being held and rates of being released and proportions black defendants make up of persons held and persons released – violent offenses.

 

Risk

Level Below

Number

Blacks Below

Number Whites Below

Prop of Blacks Below

Prop of Whites Below

Ratio

Bl/Wh

Incar 

Ratio Wh/Bl Rel Rt

Bl Prop of Incar

Bl Prop of Rel

2

368.79

650.61

20.65%

45.83%

1.46

2.22

65.99%

37.38%

3

682.93

867.07

36.32%

61.08%

1.64

1.68

68.43%

44.06%

4

954.27

1055.49

50.75%

74.36%

1.92

1.47

71.79%

47.48%

5

1181.71

1175.61

62.84%

82.82%

2.16

1.32

74.13%

50.13%

6

1364.63

1264.02

72.57%

89.05%

2.50

1.23

76.84%

51.91%

7

1550.61

1332.93

82.46%

93.90%

2.88

1.14

79.21%

53.77%

8

1685.37

1377.44

89.62%

97.04%

3.50

1.08

82.26%

55.03%

9

1768.29

1392.68

94.03%

98.11%

3.16

1.04

80.70%

55.94%

10

1846.95

1414.02

98.22%

99.61%

4.61

1.01

85.94%

56.64%

 

Like Table 1, Table 2 shows a general pattern whereby the more lenient the standard for release, the greater is the relative difference in rates of incarceration and the smaller is the relative difference in rates of release.  Also, as with Table 1, the more lenient the standard for release the larger is the proportion blacks make up both of defendants incarcerated and defendants released.  There is, however, a departure from the general pattern in the penultimate row, which, compared with the prior row, shows a decrease in the relative difference in rates of being incarcerated and a decrease in the proportion blacks make up of incarcerated persons.  Such departure is a function of an irregularity in the distributions at a point where there are comparatively few observations, especially for whites, or (probably less likely) inexactness in my estimations of numbers reflected by each bar in the bar charts.

One will note that at each cut point the relative difference in rates of being incarcerated are larger, while relative differences in rates of being released are smaller for violent offenses than for all offenses.  Such a pattern is to be expected given that risks of violent recidivism are much lower than risk of any recidivism (which is not to say that the relationship of the black and white risk distributions for violent offenses is the same as for all offenses[vi]). 

Other data in the ProPublica article also reflect aspects of the pattern described at the outset.  One corollary to the pattern is that a factor that affects outcome rates for two groups with different baseline rates for the outcome will tend to cause a larger proportionate change in the outcome rate for the group with the lower baseline rate for the outcome while causing a larger proportionate change in the opposite outcome rate for other group.  See "Race and Mortality Revisited" (at 340-341) and the Subgroup Effects and Subgroup Effects – Nonclinical pages.  Given that whites have lower recidivism rates among persons designated low risk, having a higher risk rate will tend to cause larger proportionate increases in recidivism for whites than blacks while causing larger proportionate decrease in rates of not recidivating for blacks than whites.  The data in the ProPublica two-by-two tables for general recidivism and for violent recidivism (which treat all risks ranges of five or higher as high risk) show such a pattern.  That is, for recidivism generally the ratio of the recidivism rate for the high risk group (decile 5 or above) to the low risk group (decile 4 or below) was 2.05 for whites and 1.80 for blacks, while the ratio of the rate of no recidivism for the low risk group to the high risk group was 1.74 for whites and 1.76 for blacks; for violent recidivism the ratio of the recidivism rate for the high risk group to the rate for the low risk group was 2.36 for whites and 1.27 for blacks, while the ratio of the rate of no recidivism for the low risk group to the rate for the high risk group was 1.12 for whites and 1.14 for blacks

As reflected in the references, however, this pattern is entirely misunderstood and observers mistakenly attribute significance to differing relative effects on whichever outcome they happen to be examining.  Comparing deciles 8 to 10 with deciles 1 to 3, the ProPublica article notes that ratios of recidivism rates of the higher risk group to the lower risk group are 3.61 for whites and 2.99 for blacks.  Comparing deciles 5 to 7 with deciles 1 to 4, the article notes that ratios of recidivism rates of the higher risk group to the lower risk group are 2.23 for whites and 1.95 for blacks.  Such patterns are in accord with the pattern just described and the underlying data would almost certainly show that being in the higher risk category reduces the rates of avoiding recidivism proportionately more for blacks than whites.

The ProPublica article, however, reads the larger relative increase in recidivism for whites than blacks as if it were significant, noting that it shows that the score is performing differently for the two racial groups.  This statement reflects the illogical expectation that in the normal course a factor that affects outcome rates will cause proportionate changes in the outcome rates for group with different baseline rates.  The expectation is illogical because it is mathematically impossible for a factor to cause equal proportionate changes in different baseline rates for an outcome while causing equal proportionate changes in the opposite outcome.  See references in the last paragraph and the  Illogical Premises page. 

The data in the two-by-two tables for general recidivism and for violent recidivism also show that, both among persons designated high risk and persons designated low risk, blacks are more likely to recidivate than whites.  The bar charts illustrate why that it is to be expected.  That is, within each broad risk category, blacks have higher risk scores than whites.

 



[i] Whether an instrument regards the favorable outcome or the adverse outcome as a positive is entirely arbitrary.

[ii] With regard to assumptions in the 1996 article about effects of modifications to California’s Three Strikes law on measures of racial disparity see the California Prison Population page.  It shows how the assumption is supported by data in a 2004 study of California’s prison population.

[iii] The fractional persons in the text supra and in the table infra reflect that the numbers were estimated from bar charts.

[iv] I am not sure why the article calls each level a decile.  But I employ that usage in certain places here.

[v] See note 40 at page 20 Comments for the Commission on Evidence-Based Policymaking (Nov. 14, 2016) regarding my preference for using the larger figure in the numerator or a ratio used to represent a relative difference. 

[vi] Readers should keep in mind that observed patterns of relative differences (and other measures) are functions of both the differences in the distributions and the overall prevalence of the outcome.