Solutions: Measures of Health and Healthcare Disparities that are Unaffected by the Prevalence of an Outcome (Sep. 20, 2008; rev. Feb. 22, 2009)
Note of February 22, 2010: The material below discusses a method of measuring disparities that derives from a pair of rates the means between the underlying risk distributions and the Solutions Database page provides a database with which to implement that approach I note here that it recently came to my attention that the same result can be achieved by a probit analysis, such as may be implemented from a downloadable Excel file made available by David B. Wilson of George Mason University (http://mason.gmu.edu/~dwilsonb/ma.html).
The hundred plus references on the Measuring Health Disparities page (MHD) of jpscanlan.com and the content of the Scanlan’s Rule Scanlan’s Rule page of the same site mainly address the reasons that the standard measures of differences between rates of experiencing an outcome – relative differences (in an outcome or its opposite), absolute differences, odd ratios – are not useful for evaluating the size of demographic differences. Such reason is that each measure tends to change solely because of a change in overall prevalence. Such patterns are illustrated with income data, hypothetical test score data, and NHANES data on systolic blood pressure. Item B7 (BSPS 2006 Paper) also explains that the use of differences in longevity is problematic for the same reason. That is, longevity differences, whether measured in absolute or relative terms, tend to change as overall mortality rates change. Item D43 (Comment on Boström) illustrates the same point with regard to the Gini coefficient and Concentration Index sub-page of MHD illustrates it with regard to the concentration index. Thus, various references, especially A1 (Chance 2006) and B7, raise an issue as to whether one can actually measure health disparities with sufficient reliability to justify the resources devoted to such study. See also Section A.6 of Scanlan’s Rule.
Several items (e.g., B3, B9-B11) explore the possibility of using genuinely continuous measures, assuming that they do not raise the same issues as binary measures, and in doing so, explain why many seemingly continuous measures are in fact functions of dichotomies and hence implicate the same interpretive issues as binary measures.
As discussed in many places, the essential criterion of an effective measure of differences between rates is that the measure does not change when there occurs solely a change in overall prevalence of an outcome akin to that effected by the lowering of a test cutoff or a reduction in poverty such as to enable everyone just below the poverty line to escape poverty. That way, when the measure does change, one knows something meaningful has occurred. A number of more recent works explore the possibility of measuring the size of disparities in particular settings by deriving, from the rates of the advantaged and disadvantaged groups being compared in each setting, the size of the difference between means of hypothesized underlying distributions of continuously-scaled risks of experiencing an outcome (with such difference measured in terms of percentage of a standard deviation). Thus, for example, if in one setting the advantaged groups’ rate of experiencing some adverse outcome is 20.0% and the disadvantaged group’s rate of experiencing such outcome is 36.7%, those rates would suggest a difference between hypothesized means of .5 standard deviations. If in another setting the advantaged group’s rate is 14.9% and the disadvantage group’s rate is 23.0%, those rates would suggest a difference between means of .3 standard deviations. The on-line comments and conference presentation exploring this approach are set out in section A and B several paragraphs below. As a rule, the references terms the difference between means identified in this manner as the EES, for estimated effect size. In a few cases, I have used other terms, such as HMD for hypothetical mean difference (as in Comment on Moser) or EDM for estimated difference between means (as in Comment on Hetemaa). Further, the Subgroup Effects sub-page of Scanlan’s Rule page uses the term EES-Alpha to identify a difference between the outcome rates of two groups and EES-Beta to identify the effect of a factor on an outcome rate of any group.
As repeatedly noted in those works (and as explored at greatest length in Section B of reference D43) (Comment on Bostrum)), this approach involves some speculation, given that we do not know what the underlying distributions actually look like. And, as discussed in references D43 and D46a (Comment on Werner) the approach appears not to be theoretically sound as to situations where we know the distributions at issue are not normal because they are in fact truncated portions or larger distributions even when the larger distributions are perfectly normal. This point is illustrated in Table 7 of Comment on Bostrum and further discussed and illustrated in the Truncation Issues sub-page of Scanlan’s Rule page of jpscanlan.com. See also the treatment of a related issue in the Cohort Considerations sub-page of MHD. Nevertheless, an approach along these lines is superior to anything else currently employed.
In previous versions of this item, I have discussed issues about the calculation of confidence intervals, a matter raised by attendees at various conference presentations. In light of my coming to recognize that the approach described here is achieves the same results as a probit analysis, it would appear that the confidence intervals are those derived by a probit analysis.
I have done those calculations in a mechanical manner. That is, I have created a table based on 100 iterations of widely available data on proportions of populations falling above certain points on a distribution. The values are staggered to reflect differences ranging from .01 to 1.00 standard deviations between means. The estimated differences derived by programmatically identifying, for every pair of rates an advantaged and disadvantaged group that is of interest, the closest match in the table to each such rate.
A copy of the table, as a zipped text file, is available at LINK. Rates are adverse outcome rates. A complete database is available at Solutions Database.
I assume that statisticians who can derive the EES by means of a formula could also devise a means of calculating confidence intervals. I would caution, however, against placing too much weight on such calculations, since they would necessarily be based on assumptions about the shape of the underlying distributions. An approach that may be of some value for purposes of determining whether a change over time is in fact other than the standard consequence of a change in prevalence would be to determine whether the actual disadvantaged group’s rate in the later period differs to a statistically significant degree from a rate that would be consistent with no change. That is, Table 1 of B17 (BSPS 2008) shows that when the favorable outcome rate of the advantaged group (AG) is 40% and the favorable outcome rate of the disadvantaged group (DG) is 23%, the EES is .50. It also shows that in a later period, when AG’s rate is 58% and DG’s rate is 39%, such rates would be consistent with the same .50 EES. Table 2 then shows that if DG’s rate in the later period were in fact 40%, the EES would have declined to .47. Thus, the issue of whether the change in the EES from .50 to .47 was statistically significant would initially involve whether the difference between AG rates of 39% and 40% is statistically significant (allowing that there also would issues concerning confidence intervals on the initial figures). All to say, determining reliable confidence intervals may be an issue of some complexity and likely involve some degree of uncertainty. Again, however, these issues regarding the proposed approach do not make it inferior to an approach that simply examines a measure of differences between rates that tend to be affected by the prevalence of an outcome without regard to the affect of overall prevalence.
Note added Jan. 6, 2008: In a variety of places, I have noted the implications of irreducible minimums both with respect to standard patterns of changes of relative differences and the approach embodied in the Solutions Database. Further attention to the irreducible minimum issue with regard to item D63 (Victora Comment), prompted me to give the matter further attention and to modify the Solutions Database to allow one to address the issue. Discussion of the role of irreducible minimums as it bears on the database may be found on the Irreducible Minimums sub-page of MHD. The instructional narrative of the Solutions Database sub-page has also been modified.
On-line comments and conference presentations illustrating the approach are listed in the sections that follow: Illustrations may also be found in Tables 1 and 2 of the Mortality and Survival sub-page of the Scanlan’s Rule page.
A. On-line Comments
D43. Comparing the size of inequalities in dichotomous measures in light of the standard correlations between such measures and the prevalence of an outcome. Journal Review Jan. 14, 2008 (responding to Boström G, Rosén M. Measuring social inequalities in health – politics or science? Scan J Public Health 2003;31:211-215):
http://journalreview.org/v2/articles/view/12850975.html
(Version with properly formatted tables: http://www.jpscanlan.com/images/Bostrom_and_Rosen_Comment.pdf)
D45. Comparing health inequalities across time and place with an understanding of the usual correlations between various measures of difference and overall prevalences. Journal Review Jan. 30, 2008 (responding to Moser K, Frost C, Leon D. Comparing health inequalities across time and place—rate ratios and rate differences lead to different conclusions: analysis of cross-sectional data from 22 countries 1991–200. Int J Epidemiol 2007;36:1285-1291): http://journalreview.org/v2/articles/view/17898027.html
D46. Pay-for-performance implications of the failure to recognize the way changes in prevalence of an outcome affect measures of racial disparities in experiencing the outcome. Journal Review Feb. 8, 2008 (responding to):
http://journalreview.org/v2/articles/view/15769766.html
D46a. Implications of the focus on racial/ethnic disparities in control rather than processes in the context of pay-for-performance . Journal Review Feb. 10, 2008 (further response to Werner, RM, Asch DA, Polsky D. Racial profiling: The unintended consequences of coronary artery bypass graft report cards. Circulation 2005;111:1257–63):
http://journalreview.org/v2/articles/view/15769766.html
D48. Perceptions of changes in healthcare disparities among the elderly dependant on choice of measure, Journal Review 2/12/08 (responding to Escarce JJ, McGuire TG. Changes in racial differences in use of medical procedures and diagnostic tests among elderly persons: 1986-1997. Am J Public Health 2004;94:1795-1799):
http://journalreview.org/v2/articles/view/15451752.html
D52. Study illustrates ways in which the direction of a change in disparity turns on the measure chosen. Pediatrics Mar. 27, 2008 (responding to Morita JY, Ramirez E, Trick WE. Effect of school-entry vaccination requirements on racial and ethnic disparities in Hepatitis B immunization coverage among public high school students. Pediatrics 2008;121:e547-e552):http://pediatrics.aappublications.org/cgi/eletters/121/3/e547
D53. Comparisons of the sizes of differences between black and white rates for different procedures are not informative without consideration of the overall levels for each procedure. Journal Review Mar. 28, 2008 (responding to Baicker K, Chandra A, Skinner JS, Wennberg JE. Who you are and where you live: how race and geography affect the treatment of Medicare beneficiaries. Health Affairs 2004:Var-33-Var-44):
http://journalreview.org/v2/articles/view/15471775.html
D55. Understanding patterns of absolute differences in vaccination rates in different settings. Journal Review Apr. 22, 2008 (responding to Schneider EC, Cleary PD, Zaslavsky AM, Epstein AM. Racial disparity in influenza vaccination: Does managed care narrow the gap between blacks and whites? JAMA 2001;286:1455-1460):
http://journalreview.org/v2/articles/view/11572737.html
D56. Study shows different adjustment approaches rather than different relative and absolute perspectives. Journal Review May 1, 2008 (responding to Khang YH, Lynch JW, Jung-Choi K, Cho HJ. Explaining age-specific inequalities in mortality from all causes, cardiovascular disease and ischaemic heart disease among South Korean public servants: relative and absolute perspectives. Heart 2008;94:75-82: http://journalreview.org/v2/articles/view/17591645.html
D58. Identifying meaningful differences in inequalities in revascularization rates in different settings. Journal Review May 9, 2008 (responding to Hetemaa T, Keskimäki I, Manderbacka, et al. How did the recent increase in the supply or coronary operations in Finland affect socioeconomic and gender equity in their use? J Epidemiol Community Health 2003;57:178-185): http://journalreview.org/v2/articles/view/12594194.html
D60. Illustrating whether the relationship between race and allostatic load scores increases with age. Journal Review July 24, 2008 (responding to Geronimus A, Hicken M, Keene D, and Bound J. Weathering and Age Patterns of Allostatic Load Scores Among Blacks and Whites in the United States. Am J Pub Health 2006;96:826-833):http://journalreview.org/v2/articles/view/16380565.html
D62. Interpreting patterns of changes in measures of demographic differences in folate status in light of overall improvements in folate status. Journal Review Dec. 2, 2008 (responding to Dowd JB, Aiello AE. Did national folic acid fortification reduce socioeconomic and racial disparities in folate status in the US. Int J Epidemiol 2008:37:1059-1066): http://journalreview.org/v2/articles/view/18456713.html
D69. Study raises a number of issues about analyzing disparities between and among demographic groups. Journal Review March 23, 2009 (responding to Harper S, Lynch J, Meersman SC, et al. Trends in area-socioeconomic disparities in breast cancer screening, mortality, and survival among women ages 50 years and over (1987-2005). Cancer Epidemiol Biomarkers Prev 2009;18(1):121-131): http://journalreview.org/v2/articles/view/19124489.html
D82. Importance of distinguishing disparities in survival from disparities in mortality. Feb. 17, 2010 (responding to Keegan, THM, Clarke CA, Chang ET, et al. Disparities in survival after Hodgkin lymphoma: a population based study. Cancer Causes Control 2009;20:1881-1892: http://www.jpscanlan.com/images/Comment_on_Keegan.pdf
B. Conference Presentations
B13. Can We Actually Measure Health Disparities?, presented at the 7th International Conference on Health Policy Statistics, Philadelphia, PA, Jan. 17-18, 2008 (invited session).
Abstract: http://www.amstat.org/meetings/ichps/2008/index.cfm?fuseaction=AbstractDetails&AbstractID=300283
PowerPoint Presentation: http://www.jpscanlan.com/images/2008_ICHPS.ppt
Oral Presentation: http://www.jpscanlan.com/images/2008_ICHPS_Oral.pdf
B14. Measuring Health Disparities, presented at the Kansas Department of Health and Environment, Center for Health Disparities, 2008 Health Disparities Conference, Topeka, Kansas, Apr. 1, 2008.
PowerPoint Presentation: http://jpscanlan.com/images/KDHE_Presentation.ppt
B15. Measures of Health Inequalities that are Unaffected by the Prevalence of an Outcome, presented at the 16th Nordic Demographic Symposium, Helsinki, Finland, June 5-7, 2008.
PowerPoint Presentation: http://jpscanlan.com/images/Scanlan_JP_NDS_Presentation_2R.ppt
B16. Evaluating The Sizes Of Differences Between Group Rates In Settings Of Different Overall Prevalence, presented at the Joint Statistical Meetings of the American Statistical Association, International Biometric Society, Institute for Mathematical Statistics, and Canadian Statistical Society, Denver, Colorado, Aug. 3-7, 2008.
Abstract: http://www.amstat.org/meetings/jsm/2008/onlineprogram/index.cfm?fuseaction=abstract_details&abstractid=300998
Power Point Presentation:
http://jpscanlan.com/images/jsm 2008.ppt
B17. An Approach to Measuring Differences Between Rates that is not Affected by the Overall Prevalence of an Outcome, presented at the British Society for Populations Studies Conference 2008, Manchester, England, Sept. 10-11, 2008.
PowerPoint Presentation: http://jpscanlan.com/images/BSPS_2008_Presentation_2.ppt
B18. Approaches to Measuring Health Disparities that are Unaffected by the Prevalence of an Outcome, presented at American Public Health Association 136th Annual Meeting & Exposition, San Diego, California, Oct. 25-29, 2008.
Abstract: http://apha.confex.com/apha/136am/webprogram/Paper173021.html
PowerPoint Presentation:
http://www.jpscanlan.com/images/Scanlan_APHA_2008_Presentation.ppt
B19. Interpreting Differential Effects in Light of Fundamental Statistical Tendencies, presented at 2009 Joint Statistical Meetings of the American Statistical Association, International Biometric Society, Institute for Mathematical Statistics, and Canadian Statistical Society, Washington, DC, Aug. 1-6, 2009.
Abstract: http://www.amstat.org/meetings/jsm/2009/onlineprogram/index.cfm?fuseaction=abstract_details&abstractid=304941
PowerPointPresentation : http://www.jpscanlan.com/images/Scanlan_JSM_2009.ppt
Oral Presentation: http://www.jpscanlan.com/images/JSM_2009_ORAL.pdf
B.20. Measuring Health Inequalities by an Approach Unaffected by the Overall Prevalence of the Outcomes at Issue, presented at the Royal Statistical Society Conference 2009, Edinburgh, Scotland, Sept. 7-11, 2009.
Abstract: http://www.jpscanlan.com/images/RSS_2009_Abstract.pdf
PowerPoint Presentation: http://www.jpscanlan.com/images/Scanlan_RSS_2009_Presentation.ppt
|