James P. Scanlan, Attorney at Law

Measurement Letters

(rev. Aug. 23, 2026) 

Listed below are letters and other communications since 2009 explaining to institutions or organizations – with respect to the recipients’ own analyses of demographic differences, analyses by others that pertain to the recipients, or guidance on analyses of demographic differences that the recipients have given (or should give) to others – that virtually all analyses of demographic differences regarding outcome rates are fundamentally undermined by failure to recognize patterns by which measures tend to be affected by the prevalence of an outcome.  One such patterns (though not the only important one) is that whereby when two groups differ in their susceptibility to an outcome, the greater tends to be the relative difference between the rates at which the groups experience the outcome and the smaller tends to be the relative differences between rates of avoiding the outcome, an pattern I have explained in hundreds of places since 1987.  One obvious example of the pattern may be found in the way that lowering a test cutoff and thereby increasing test passage rates and reducing test failure rates, while tending to REDUCE relative differences between rates at which higher- and lower-scoring groups pass the test, tends to INCREASE relative differences between the rates at which the groups fail the test.  The fact that lowering a test cutoff tends to reduce relative differences in pass rates is quite well known (as is, presumably, that raising the cutoff and thereby reducing test passage rates would tend to increase such differences). But the fact that lowering the cutoff tends to increase relative differences in pass rates is virtually unknown, as the fact that fact that reducing any adverse outcome tends to increase relative differences between rates at which advantaged and disadvantaged groups experience the outcome.  In fact, at least among persons and entities whose activities importantly involve the analysis of demographic differences (including federal, state, and local governments), there exists a near universal belief that reducing an adverse outcome will tend to reduce, rather than increase, relative differences between the rates at which advantaged and disadvantaged groups experience the outcome. As discussed in “The Mismeasure of Health Disparities,” Journal of Public Health Management and Practice (July/Aug. 2016), “Race and Mortality Revisited,” Society (July/Aug. 2014), as of 2010, an exception might have been found among statisticians at the National Center for Health Statistics (NCHS). But, more likely, assuming that agency ever actually understood the matter in an institutional sense, it no longer does so.  

In any case, many items since 2012 are principally focused on the mistaken belief that generally reducing some adverse outcome will tend to reduce relative demographic differences in rates of experiencing the outcome (and the proportions more susceptible groups make up of persons experiencing the outcome) and explain that the opposite is the case.  Some treatments of the mistaken belief that generally reducing adverse criminal justice and school discipline outcomes will tend to reduce relative racial/ethnic differences in rates of experiencing the outcomes, and the proportion racial/ethnic minorities make up of persons experience the outcomes, may be found in “Maryland Discipline Study Shows Usual – But Misunderstood – Effects of Policies on Measures of Racial Disparity “Usual, But Wholly Misunderstood, Effects of Policies on Measures of Racial Disparity Now Being Seen in Ferguson and the UK and Soon to Be Seen in Baltimore,” Federalist Society Blog (Dec. 4, 2019), “COPAA v. DeVos and the Government’s Continuing Numeracy Problem,” Federalist Society Blog (Sept. 12, 2019),  “United States Exports Its Most Profound Ignorance About Racial Disparities to the United Kingdom,” Federalist Society Blog (Nov. 2, 2017), “The misunderstood effects of the Baltimore police consent decree,” The Daily Record (Feb. 15, 2018), and “Discipline disparities in Md. Schools,” Daily Record (June 21, 2018).

As of the July 2026 updating of this page, one of the listed communications involves an issue that is somewhat different from that addressed in the others (though it similarly involves the failure to understand implications of the shapes of normal risk distributions).  This is the June 28, 2026 communication to the Department of Justice concerning its investigation of Yale University School of Medicine where the agency found that certain minority groups were being favored on the basis that admitted students from those groups had weaker qualifications than admitted students from other groups. The communication explains that, when one group has average weaker qualifications among applicants for some position than another group, the former typically will have weaker average qualifications among persons selected for the position than the other group, even when all applicants are treated equally. Hence, it is not possible to infer that the group with weaker qualifications among applicants was favored in a selection process based solely on the fact that it has average weaker qualifications among persons selected.  See “Illusions of Job Segregation” Public Interest (Fall 1988) (discussion of Table IV); “The Perils of Provocative Statistics,” Public Interest (Winter 1991) (at 13); "Perils of Using Statistics to Show Presence or Absence of Loan Bias," American Banker (Jan. 3, 1997);  "Both Sides Misuse Data in the Credit Discrimination Debate," American Banker (July 22, 1998).

***

Email to DOJ Counsel in Thakur v. Trump and New Jersey v. OMB (Aug. 3, 2026)

Online Communication to Department of Justice Regarding Agency’s Letter of Findings on Discrimination at Yale University School of Medicine (June 28, 2026)

Online Communication to Department of Justice Regarding Agency’s Memorandum Opinion on Uniform Guidelines for Employee Selection Procedures (June 28, 2026)