James P. Scanlan, Attorney at Law

Comparing Averages

(May 15, 2010)

Implicit in the patterns described on the Scanlan’s Rule page is a fundamental problem with a comparison of average outcome rates. 

Table A below is an abbreviated version of Table 1 of the 2006 British Society for Population Studies Presentation.  The BSPS table shows the implications of overall prevalence with regard to various measures of differences in outcome rates.  Specifically, it shows the way that relative differences in an adverse outcome, relative differences in the opposite (favorable)  outcome, absolute difference between rates, and odds ratios tend to change systematically as the overall prevalence of an outcome changes.

Think of two rows in Table A as involving two equally sized subpopulations of the advantaged and disadvantaged  groups.  One might think of it in terms of failure rates on a test for high-education and low-education subpopulations of the two groups or mortality rates for persons under and over, say, age 45 for the two groups.  Typically, in comparing the overall outcome rates for the two groups, one would compare the overall averages for each group.  One might weight averages according to the size of the subpopulations or adjust for any difference in the distribution of the two groups in the subpopulations.  But the specifications set out in the first sentence of this paragraph render such actions unnecessary.

By definition, the difference between each group’s rate in the subpopulations is half a standard deviation (as shown in the final column).   As discussed generally on the Scanlan’s Rule page and Solutions sub-page of the Measuring Health Disparities page, that is the only sound measure of differences between the rates because it is the only measure unaffected by the overall prevalence of an outcome.  And given that there is no difference in the distributions of the two groups according to subpopulation, there is seems to be no reason to regard the difference between the overall averages to be other than .5 standard deviations.   But Table A shows that EES derived from the averages is .44 standard deviations.

The above suggests that comparisons of overall average should be based on the appropriately adjusted averages of the EES figures derived for the subpopulations.  See related discussion in Section 5 of the Adjustment Issues sub-page of the Vignettes page of jpscanlan.com