James P. Scanlan, Attorney at Law

Criminal Record Effect

(May 14, 2013; rev. May 27, 2013)

Prefatory note:  This subpage is related to the many pages and subpages of this site addressing the statistical pattern whereby the rarer an outcome the greater tends to be the relative difference in experiencing it and the smaller tends to be the relative difference in avoiding it.  It is most closely related to the Subgroup Effects subpage of the Scanlan’s Rule page, which addresses the problematic nature of standard subgroup analyses that are based on the assumption that, absent a subgroup effect, a factor will have the same relative effect on different baseline rates for an adverse outcome  even though one would commonly reach an opposite conclusions as to the nature of the subgroup effect if one examined relative effects on the corresponding favorable outcome.  It is also related to the Mortality and Survival page, which discusses the common pattern of referring to racial and other disparities in survival rates while actually analyzing relative differences in mortality rates, even though analyzes of  mortality rates tend to yield opposite conclusions from analyses of survival rates on mortality rates. The studies discussed here reflect the opposite situation where subgroup analyses examine relative effects on the favorable outcome.  The studies also reflect instances where the authors draw conclusions based on perceptions about the comparative size of proportionate changes, thus highlighting the incongruity of the National Center for Health Statistics view that one may reasonably interpret data on group differences by simply choosing between interpretations based on relative differences in favorable outcomes and relative differences in adverse outcomes.

Section A discusses the common practice of drawing conclusions about the comparative size of relative differences or the comparative size of relative effects by examining adverse outcome rates without recognizing that one would commonly reach opposite conclusions based on the examination of the corresponding favorable outcome rates.  Section B discusses studies of the comparative size of the effects of having criminal records on black and white employment prospects that examined favorable outcome rates.

A.  Background – Commonplace Misguided Comparisons of Relative Differences in (Relative Effects on) Adverse Outcomes 

There exists a great deal of research maintaining that advantaged groups are more affected by some beneficial or deleterious characteristic than disadvantaged groups.  For example, it has been in various ways noted that whites benefit more than blacks from being of high socioeconomic, income, education status (a pattern that is often discussed in terms of larger racial disparities in adverse outcome rates among subpopulations of higher compared with lower socioeconomic, income, education status, which is a corollary to a pattern whereby being of higher status reduces the adverse outcome proportionately more among whites than among blacks as well as the corresponding pattern by which being of lower status increases the adverse outcome more among whites than blacks).  I discuss examples of such research in “Can We Actually Measure Health Disparities?” (Chance, Spring 2006), “Race and Mortality” (Society, Jan/Feb 2000), “The Perils of Provocative Statistics” (Public Interest, Winter 1991), and address such research at pages 17-19 of the Harvard University Measurement Letter.  Similarly, it has been maintained that racial and socioeconomic  health disparities are greater among the young than the old, which is the same as maintaining that being young reduces adverse health outcomes, or that being old increases adverse health outcomes, more for advantaged groups than disadvantaged groups.[i] 

The so-called reporting heterogeneity research is much to the same effect.  It finds, for example, that having a chronic condition will reduce an adverse health appraisal (commonly identified as health-less-than-good) proportionately more for higher than lower socioeconomic groups.  I discuss this research on the Reporting Heterogeneity of the Measuring Health Disparities page of jpscanlan.com.

Such research has examined relative effects on adverse outcomes.  And, as I have explained at varying length in each of the above references, the research is invariably flawed for failing to recognize that the factor will tend to show a larger proportionate effect on the adverse outcome rates of the advantaged population than the disadvantaged population simply because the adverse outcome rate is lower in the advantaged population.  Almost invariably the factor will also show a larger proportionate effect on the favorable outcome rate in the disadvantaged group.  That is, for example, being of high socioeconomic status will tend to reduce mortality more for whites than blacks, while increasing survival more for blacks than whites; having high income will tend to reduce mortgage rejection rates more for whites than black, while increasing mortgage approval rates more for blacks than whites (see the Disparities – High Income subpage of the Lending Disparities page).   The Life Tables Illustrations subpage of the Scanlan’s Rule page (SR) and the Life Table Information Document nicely illustrate these patterns in comparisons of rates of dying before and living beyond certain ages of  white men versus white women, black men versus black women, black men versus white men, and black women versus white women, where the former group is the disadvantaged group in each pair.  The data show that relative differences in mortality tend to decrease with age, while relative differences in survival tend to increase with age. 

These and like patterns are discussed and illustrated in various works made available on the Measuring Health Disparities page (MHD) and in the narrative material on MHD and many other pages of jpscanlan.com.  Hence, had the researchers who find that a factor has a greater effect on an advantaged group than a disadvantaged group examined the effects on the favorable outcome, they would commonly have found the factor to have a larger proportionate effect on the disadvantaged group than the advantaged group. 

Such patterns are also the subject of Subgroup Effects and Illogical Premises subpage of SR, which discuss the view underlying standard subgroup analyses that absent the occurrence of something significant a factor that affects outcome rate will tend to cause equal proportionate changes in different baseline rates.  Those pages explain that such views is not only incorrect, but illogical (given that a factor cannot cause equal proportionate changes in different baseline rates for an outcome while causing equal proportionate changes to the opposite outcome rates).   See also the Explanatory Theories subpage of SR, which discusses the way that observers devise theories to explain observed patterns, and pages 39-43 of the Harvard letter, which discuss inferences observers draw based on observed patterns, without recognizing that, solely due to the shapes of the underlying distributions, the patterns are to be expected in the circumstances or that examination of relative effects on the opposite outcome commonly would yield opposite conclusions about the implications of the patterns. 

But while the above research has generally focused on relative effects on adverse outcomes, there also exists a smaller body of research that examines favorable outcomes, with the tendency therefore to find that a factor will have a larger effect on the disadvantaged group, which has the lower baseline rate.  A good part of such research is that which examined healthcare disparities and found that as rates of beneficial health procedures increased relative differences in receipt of such procedures decreased, which I discuss in “Race and Mortality” and “Can We Actually Measure Health Disparities.”  The latter item and the Comment on Morita Pediatrics 2008, as well pages 29-31 of the Harvard letter, discuss, the disarray in health disparities research as a result of the National Center for Health Statistics recommendation that all health and healthcare disparities be measured in terms of relative differences in adverse outcome, which recommendation, if followed, will tend to cause healthcare disparities that had been regarded as decreasing now to be regarded as increasing.[ii]

B.  Studies of Comparative Relative Effects of Criminal Records on Favorable Outcome Rates of Whites and Blacks

A number of studies recently coming to my attention involving the examination of relative differences in favorable outcome rates (and doing so in terms of differences in the relative effects of a factor on favorable outcome rates) have addressed the effects of criminal records on employment prospects of black and white applicants.  In the first study, [iii]  based on the fact that among black and white tester pairs of applicants that did or did not indicate on their application that they had a criminal conviction, the proportionate effect of indicating that the applicant had a conviction on chances of receiving a callback was greater for the black tester pairs than the white tester pairs, the author concluded that the effect of a conviction was greater for blacks than whites.[iv]   The callback rates are set out in Table 1, along with the black and white (a) ratios of the callback rates of applicants without convictions to the callback rates of applicants with convictions and (b) ratios rates of failing to receive of applicants with a conviction with a conviction to the rates of failing to receive a callback of applicants without a conviction.[v]   The table also shows the EES (estimated effect size), which, as explained in the Subgroup Effects subpage (and pages 24-28 of the Harvard letter), is the only sound measure of the strength of the effect of the conviction on each group.

Table 1.  Rates of Receiving Callbacks for Testers Indicating a Conviction and Not Indicating a Conviction on their Applications by Race of Tester, with Rate Ratios for Receipt and Non-Receipt of Callbacks and Estimated Effect Size (from Pager 2003) 

The table shows the common pattern whereby observers who rely on relative effects with respect to the favorable outcome would find blacks (the group with the lower baseline callback rate) to be more affected than whites, while observers who rely on relative effects with respect to the adverse outcome would find whites (the group with the lower baseline no-callback rate) to be more affected than blacks.  The EES figures reveal that any difference was very small.[vi]

As suggested above, one would observe the same sort of pattern in terms of the differing sizes of relative differences between blacks and whites with conviction and without convictions.  That is, as shown in Table 1a, the relative difference between white and black rates of callback is greater for applicants with a conviction (where callback is rarer) than applications without a conviction, while the relative difference between white and black rates of not receiving a callback is greater for applicants without a conviction than with a conviction.  But, again, the difference between the EES figures is very small.   As discussed in note 8 of the Subgroup Effects subpage, the difference between the EES figures for a factor’s effect on the blacks and whites will equal the difference between the EES figures for the effect of race on applicants with and without a conviction.

Table 1a.  Black and White Rates of Receiving Callback by Whether Applicant Indicated a Conviction, with Rate Ratios for Receipt and Non-Receipt of Callbacks and Estimated Effect Size (from Pager 2003) 

While not precisely germane to the instant discussion, I nevertheless note that the author of the study co-authored a later tester study of the same type, [vii] with results that the author regarded as similar to those of the former study, but which were different in an important respect.  These results are shown in Tables 2 and 2a, along with the other information shown in Table 1 and 1a.

Table 2.  Rates of Receiving Callbacks for Testers Indicating a Conviction and Not Indicating a Conviction on their Applications by Race of Tester, with Rate Ratios for Receipt and Non-Receipt of Callbacks and Estimated Effect Size (from Pager et al. 2009) 

In this case, one observes not only that the relative effect on callback rates was greater for blacks than whites, but that the relative effect on rates of not receiving a callback was greater for blacks than whites as well.  From the latter fact, one may surmise that the effect of having a conviction was greater for blacks than whites in a meaningful way and that the difference in the size of the effect was sufficient to cause the standard statistical pattern not to be evident.  One can more efficiently reach the same conclusion based on the EES, which is considerably greater for blacks than whites.  In any case the authors' conclusions that the effect of a conviction on the employment opportunities was considerably greater for blacks than white would appear correct.[viii]  But it is necessary to understand the patterns by which measures tend to be affected by the prevalence of an outcome to reach such conclusion in a statistically sound manner.

Table 2a presents the information study in accordance with the perspective shown in Table 1a.  It shows that relative differences in between black and white rates of both favorable and adverse outcome are greater for those with convictions than without and that the EES is much greater for those with convictions.  As discussed, these are simply corollaries to the patterns shown in Table 2. [ix]

Table 2a.  Black and White Rates of Receiving Callbacks by Whether Applicant Indicated a Conviction, with Rate Ratios for Receipt and Non-Receipt of Callbacks and Estimated Effect Size (from Pager et al. 2009) 

Finally, I note while I find tester studies often provide the most persuasive evidence of discrimination, they raise a host or measurement problems.  Some of these are discussed in my "Measuring Hiring Discrimination" (Labor Law Journal, July, 1993).  Some of these affect even the EES figure.  See the discussion in the endnote attached to the third paragraph on page 26 of the Harvard letter regarding situations where applicants receive no attention at all (the fact of which of which would seem to cause EES figures to be understated).  Problems also arise when studies combine different types of job with different overall selection rates and which may also involve differences in the influence of race or criminal records.  The Comparing Averages subpage of SR merely scratches the surface of these issues.