There is a good deal of literature on differences in outcome rates that considers relative and absolute differences in outcome rates both to be legitimate ways of appraising the size of health, healthcare and other disparities, even when the measures yield opposite conclusions about the directions of changes in disparities over time or about the comparative size of two disparities in settings differentiated other than temporally. As a rule, when illustrating the way that the two measures support different conclusions, those doing so fail to note that one will also reach still different conclusions depending on whether one examines the relative difference in experiencing the outcome or the relative difference in avoiding the outcome.[i] In any case, some will discuss the choice of relative difference versus absolute difference as a measure of disparity as involving a value judgment.[ii]
As discussed in many places on this site, neither the relative difference in one outcome, nor the relative difference in the opposite outcome, nor the absolute difference is a useful indicator of the comparative size of a disparity because each is affected by the overall prevalence of an outcome. Further every one of these measures may change in one direction while there in fact occurs a meaningful change in the opposite direction. See, e.g., Measuring Health Disparities (MHD), Scanlan’s Rule, and Mortality and Survival pages. And while there can be situations where the absolute difference may be a more useful indicator of the importance of a disparity than either relative difference (see the Subgroup Effects sub-page of the Scanlan’s Rule page regarding number needed to treat), the only useful indicator of the size of disparity in a meaningful sense is an approach that derives from two rates the difference between the underlying means, as discussed on the Solutions sub-page of MHD.
The purpose of this page is to illustrate the unreasonableness, if not absurdity, of maintaining that from a relative perspective one disparity is larger while from an absolute perspective another disparity is larger. Table 1 below is derived from a table on the Case Study sub-page of the Scanlan’s Rule page where it is used to show the different perspectives from which one may view situations that reflect the same underlying difference between the circumstances of two groups. (See also Table 1 from the 2008 Joint Statistical Meetings presentation.) That is, each setting involves a situation where the means of the underlying distributions differ by half a standard deviation, but where success and failure are dichotomized at different points. AGFR and DGFR represent the favorable outcome rates for an advantaged group (AG) and a disadvantaged group (DG). The other fields show the relative differences in the favorable outcome rate, the relative difference in the adverse outcome rate, the absolute differences between failure (or success) rates, and the odds ratio.[iii] By way of further explanation with respect to the main point of this item, in Setting A, DG’s favorable outcome rate is 55% less than AG’s favorable outcome rate and 11 percentage points less than AG’s favorable (or adverse) outcome rate; in Setting B, DG’s favorable outcome rate is 43.5% less than AG’s favorable outcome rate and 17 percentage points less than AG’s favorable (or adverse) outcome rate.
|
Table 1 (from Case Study sub-page) – Illustration of Associations of Measures
of Difference Between Rates with Prevalence of an Outcome
|
|
Settings
|
AGFR
|
DGFR
|
RelDiffFav
|
RelDiffAdv
|
AbsDiff
|
OR
|
|
A
|
20.0%
|
9.0%
|
55.0%
|
13.8%
|
0.11
|
2.53
|
|
B
|
40.1%
|
22.7%
|
43.5%
|
29.2%
|
0.17
|
2.29
|
|
C
|
59.9%
|
40.5%
|
32.3%
|
48.2%
|
0.19
|
2.19
|
|
D
|
90.0%
|
78.2%
|
13.1%
|
117.1%
|
0.12
|
2.50
|
Assume that each setting involves different employers, each of whom selects from pools of applicants from the advantaged and the disadvantaged racial groups. Selection is the favorable outcome and rejection is the adverse outcome. Assume also that on average the applicants from each racial group are equally qualified and all difference in selection rates are functions of bias on the part of the employers. Now, limiting our focus to Employers A and B (for simplicity), consider which employer is more biased.
Put aside for the moment that one would reach different conclusions depending on whether one relied on relative differences in selection rates or relative differences in rejection rates. Let us first simply examine the relative differences between selection rates and the absolute differences between selection rates. Focusing on the relative difference, one would conclude that the Employer A is more biased; focusing on the absolute difference, one would conclude that Employer B is more biased. I suggest, however, that it would be manifestly absurd to maintain that both the relative and the absolute difference provide a legitimate basis for appraising the situation or to maintain that from a relative perspective Employer A is more biased and from an absolute perspective Employer B is more biased. Rather, with regard to the crucial issue of the extent to which each employer’s decisions are affected by aversion to a particular race the employers are exactly the same.
Similarly, examining the contrasting conclusions based on the two relative differences, it would be absurd to maintain that Employer A is more biased as to selection and Employer B is more biased as to rejection.
The same reasoning holds where employers A and B (or A, B, C, and D) are the same employer during different time periods and the issue is whether the degree of discrimination increased or decreased over time. The reasoning also holds where the differences in selection arise not from employer bias, but from differences in the qualifications of the two groups and the question of interest involves the degree of differences in the qualifications of the two groups that would be necessary to explain the differences in selection as other than a result of discrimination. And it holds when the decision-making process at issue is entirely objective and the issue of concern is the degree to which the selection procedures disadvantage one of the groups (in EEO parlance, which procedure has the greater disparate impact).[iv] And it holds when the issue is whether one health or healthcare disparity is larger than another or whether a health or healthcare disparity has changed over time.
[i] If one pair of rates of experiencing an outcome reflects a larger absolute difference between rates of experiencing an outcome, but a smaller relative difference between rates of experiencing the outcome, the first pair of rates necessarily will reflect a larger relative difference between rates of avoiding the outcome than the second pair of rates.
[ii] See especially the discussion in the University of Michigan CD-ROM course Measuring Health Disparities by Sam Harper and John Lynch. That item gives a good deal of attention to different interpretations of the comparative size of disparities depending on whether one examines relative or absolute differences, stating inter alia, that both measures are valid. See also Harper S, King NB, Meersman SC et al. Implicit value judgments in the measurement of health disparities. Milbank Quarterly 2010. Other work by some of these authors are discussed in the Mortality and Survival page and Comment on Harper JECH 2006 and Comment on Harper CEBP 2009.
[iii] The presentation of the relative differences is somewhat different from the way I usually illustrate relative differences (which typically involves risk ratios for the favorable outcome with AGs’ rate in the numerator and risk ratios for the adverse outcome rate with DG’s rate as the numerator). But the manner of presentation makes no differences as to the issues addressed.
[iv] But see the Employment Tests sub-page of the Scanlan’s Rule page as to reasons why lowering a test cutoff in fact reduces the racial impact of an employment test (where test scores do not determine who is selected from among persons who pass the test).