James P. Scanlan, Attorney at Law

Relative Differences Versus Absolute Differences: 

The Unreasonableness of Maintaining That One Disparity is Larger from a Relative Perspective While Another Disparity is Larger from an Absolute Perspective

(May 25, 2010; rev. Rev. Sept. 19, 2017)

Note:  This subpage was originally devoted to explaining the absurdity of maintaining that relative and absolute measures of differences between rates both provide valid appraisals of the comparative size of the difference between the circumstances of two groups even when the two measures yield opposite conclusions.  It has been expanded slightly to address relative differences in experiencing an outcome and relative differences in avoiding the outcome.  The subpage is related to the Illogical Premises sub-page of the Scanlan’s Rule page of this site, which subpage discusses that it is illogical to regard it as somehow normal that a factor that increases or decreases the likelihood of experiencing an outcome will do so to equal proportionate degrees for groups with different baseline rates for the outcome or that different baseline rates will exhibit equal proportionate changes over time (since it is impossible for two groups with different baseline  rates for an outcome to experience equal proportionate changes in rates of experiencing the outcome and equal proportionate changes in rates of failing to experience the outcome).

More recent treatment of the subject of this page may be found in the “Relative Differences and the Value Judgment Fallacy” section of “Race and Mortality Revisited,” Society (July/August 2014); Section B.1, at pages 12-15,  of “Measuring Health and Healthcare Disparities,” Proceedings of the Federal Committee on Statistical Methodology 2013 Research Conference  (March, 2014); and Section B, at page 15-19, of The Mismeasure of Discrimination” Faculty Workshop, University of Kansas School of Law ( Sept. 20, 2013).  These materials also address the fact that whenever it is noted that the absolute difference and a relative difference have changed in opposite directions, the unmentioned relative difference will have changed in the same direction as the absolute difference and opposite direction of the mentioned relative differences.  See also the “The Mismeasure of Health Disparities,” Journal of Public Health Management and Practice (July/Aug. 2016) and Comments for Commission on Evidence-Based Policymaking (Nov. 14, 2016).

I have discussed in many places that very few people analyzing demographic differences know that it is even possible for the two relative differences to change in opposite directions as the prevalence of an outcome changes, much less that more than a decade ago the National Center for Health Statistics recognized that this tends to occur systematically.  I have lately come to suspect that only a small proportion of persons analyzing demographic differences know that it is possible for the relative difference and the observer happens to be examining and the absolute difference to change in opposite directions, much less that this, too, tends to occur systematically (because observers tends to rely on the larger of the two relative differences and the absolute difference tends to change in the opposite direction of the larger relative difference as the prevalence of an outcome changes).  See my Spurious Contradictions page.  Readers inclined to share thoughts on the proportion of persons analyzing demographic difference who do not understand that it is possible for (a) the two relative differences to change in opposite direction as the prevalence of an outcome changes, or (b) the relative difference the observer happens to be examining and the absolute difference to change in opposite directions of the prevalence of an outcome change, may contact me at jps@jpscanlan.com

Some people drawn to this page by search engines may be more interested in the definitional/presentation issue treated on the Percentage Points subpage of the Vignettes page.

***

There is a good deal of literature on differences in outcome rates that considers relative and absolute differences in outcome rates both to be legitimate ways of appraising the size of health, healthcare and other disparities, even when the measures yield opposite conclusions about the directions of changes in disparities over time or about the comparative size of two disparities in settings differentiated other than temporally.  As a rule, when illustrating the way that the two measures support different conclusions, those doing so fail to note that one will also reach still different conclusions depending on whether one examines the relative difference in experiencing the outcome or the relative difference in avoiding the outcome.[i]  In any case, some will discuss the choice of relative difference versus absolute difference as a measure of disparity as involving a value judgment.[ii]

As discussed in many places on this site, neither the relative difference in one outcome, the relative difference in the opposite outcome, nor the absolute difference is a useful indicator of the comparative size of a disparity because each is affected by the overall prevalence of an outcome.  Further every one of these measures may change in one direction while there in fact occurs a meaningful change in the opposite direction.  See, e.g., Measuring Health Disparities (MHD), Scanlan’s Rule, and Mortality and Survival pages.  And while there can be situations where the absolute difference may be a more useful indicator of the importance of a disparity than either relative difference (see the Subgroup Effects subpage of the Scanlan’s Rule page regarding number needed to treat), the only useful indicator of the size of disparity in a meaningful sense is an approach that derives from two rates the difference between the underlying means, as discussed on the Solutions sub-page of MHD. 

The purpose of this page is to illustrate the unreasonableness, if not absurdity, of maintaining that from a relative perspective one disparity is larger while from an absolute perspective another disparity is larger.  Table 1 below is derived from a table on the Case Study subpage of the Scanlan’s Rule page where it is used to show the different perspectives from which one may view situations that reflect the same underlying difference between the circumstances of two groups.  (See also Table 1 from the 2008 Joint Statistical Meetings presentation.)  That is, each setting involves a situation where the means of the underlying distributions differ by half a standard deviation, but where success and failure are dichotomized at different points.  AGFR and DGFR represent the favorable outcome rates for an advantaged group (AG) and a disadvantaged group (DG).  The other fields show the relative differences in the favorable outcome rate, the relative differences in the adverse outcome rate, the absolute differences between failure (or success) rates, and the odds ratio.[iii]  By way of further explanation with respect to the main point of this item, in Setting A, DG’s favorable outcome rate is 55% less than AG’s favorable outcome rate and 11 percentage points less than AG’s favorable (or adverse) outcome rate; in Setting B, DG’s favorable outcome rate is 43.5% less than AG’s favorable outcome rate and 17 percentage points less than AG’s favorable (or adverse) outcome rate.

Assume that each setting involves different employers, each of whom selects from pools of applicants from the advantaged and the disadvantaged racial groups.  Selection is the favorable outcome and rejection is the adverse outcome.  Assume also that on average the applicants from each racial group are equally qualified and all difference in selection rates are functions of bias on the part of the employers.  Now, limiting our focus to Employers A and B (for simplicity), consider which employer is more biased.  

Put aside for the moment that one would reach different conclusions depending on whether one relied on relative differences in selection rates or relative differences in rejection rates.  Let us first simply examine the relative differences between selection rates and the absolute differences between selection rates.  Focusing on the relative difference, one would conclude that the Employer A is more biased; focusing on the absolute difference, one would conclude that Employer B is more biased.  I suggest, however, that it would be manifestly absurd to maintain that both the relative and the absolute difference provide a legitimate basis for appraising the situation or to maintain that from a relative perspective Employer A is more biased and from an absolute perspective Employer B is more biased.  Rather, with regard to the crucial issue of the extent to which each employer’s decisions are affected by aversion to a particular race the employers are exactly the same.

Similarly, examining the contrasting conclusions based on the two relative differences, it would be absurd to maintain that Employer A is more biased as to selection and Employer B is more biased as to rejection.

The same reasoning holds where employers A and B (or A, B, C, and D) are the same employer during different time periods and the issue is whether the degree of discrimination increased or decreased over time.  The reasoning also holds where the differences in selection arise not from employer bias, but from differences in the qualifications of the two groups and the question of interest involves the degree of differences in the qualifications of the two groups that would be necessary to explain the differences in selection as other than a result of discrimination.  And it holds when the decision-making process at issue is entirely objective and the issue of concern is the degree to which the selection procedures disadvantage one of the groups (in EEO parlance, which procedure has the greater disparate impact).[iv]  And it holds when the issue is whether one health or healthcare disparity is larger than another or whether a health or healthcare disparity has changed over time.