Pay-for-Performance and Healthcare Disparities
(Nov. 26, 2008; rev. Nov. 27, 2011)
Prefatory note added August 9, 2017): This page has not been materially updated since 2011. I have since then addressed that Massachusetts included a disparities element in its Medicaid pay-for-performance program that is more likely to increase rather than decrease health and healthcare disparities, among many other places, in “The Mismeasure of Health Disparities in Massachusetts and Less Affluent Places,” Department of Quantitative Health Sciences, University of Massachusetts Medical School (Nov. 18, 2015) (slides 113-118, “Race and Mortality Revisited,” Society (July/Aug. 2014) (37-39) , Committee on Statistical Methodology 2013 Research Conference, “Measuring Health and Healthcare Disparities,” Committee on Statistical Methodology 2013 Research Conference (Nov. 4-7, 2013) (24-26), Letter to Harvard University (Oct. 9, 2012) (21-23).
Prefatory note: A number of the references in this item discuss the Massachusetts program of including perceptions of the size of healthcare disparities in its Medicaid pay-for-performance program. At the time this page was first created it appeared that the Massachusetts program would measure disparities in terms of absolute differences between rates. Depending on whether the particular type of care at issue is one with generally low or generally high rates, improvements in care would tend either to increase or decrease absolute differences. Ultimately, the Massachusetts program decided to rely on the between-group variance (BGV) to measure healthcare disparities. The BGV tends to behave like the absolute difference between rates. And it would turn out that rates for the types of care examined in the program were in ranges where higher care rates tended to be associated with lower disparities. Thus, as with the absolute difference, in this setting the BGV would tend to decrease as the level of care increased, with the potential to cause the program to reward high-performing hospitals for perceived (though not necessarily actual) more equitable allocation of care. Since better performing hospitals tend to have fewer patients from disadvantaged groups, the result could well be to divert resources away from those serving large numbers of patients from disadvantaged groups and contribute to healthcare disparities. In addition, aspects of the BGV calculation tend to cause the BGV value to increase as the proportion of patients from disadvantaged groups increases, up to the point where the disadvantaged groups comprise 50% of patients. Thus, in setting where disadvantaged groups comprise less than 50% of patients, rewarding hospitals for low BGV scores would also tend to divert resources away from hospitals serving large numbers of disadvantaged patients, again, for reasons unrelated to any actual difference in equitable allocation of care within particular hospitals. This issue is addressed more fully in Between Group Variance sub-page of the Measuring Health Disparities page of this site.
In some respects, the misunderstandings concerning the measurement of health disparities detailed on the Measuring Health Disparities (MHD), Mortality and Survival, and Scanlan’s Rule (SR) pages of this site may have limited practical implications. To be sure, considerable resources are wasted in the study of disparities without recognition of the way observed patterns of differences between rates tend to be affected by the prevalence of an outcome. But the misunderstandings in the research do not have an obvious effect on health policy. That is, notwithstanding the flawed research, policymakers are likely to continue to do the usually sensible things that seem likely to improve the health both of society at large and of the disadvantaged elements of society, even if the results of such actions with regard to health disparities are almost invariably misread. It is doubtful, for example, that anyone would suggest abandoning programs like the immensely successful Back-to-Sleep program, because, contrary to expectation, it appeared to increase relative differences in SIDS rates. See reference 1 (Comment on Pickett AJPH 2005).
But one area where the measurement problems may have important practical implications involves pay-for-performance (P4P). In the United States, there is a perception that P4P may tend to increase health disparities and that, in order to counter such tendency, any P4P program should include an evaluation of effects on healthcare disparities, as is being done in Massachusetts. The perception that P4P would tend to increase healthcare disparities arose from a study finding that implementation of a CABG report card, which increased overall CABG rates, led to an increase in absolute differences between black and white CABG rates. Reference 2 (Comment on Werner Circulation 2005) addresses that study and explains why the observed increase in absolute differences between rates did not reflect an increase in disparity in a meaningful sense. References 3-5 are responses to articles relying on the study as evidence that P4P would tend to increase racial disparities in healthcare.
As explained in these references, in light of the near universal misunderstanding of the meaning of changes in each measure of differences between rates as the overall prevalence of an outcome changes, it would be a grave error to include effects on healthcare disparities as a performance criterion in a P4P program.
Let us assume that P4P will improve healthcare generally, which, in the main, will involve an increase in favorable outcome rates (including both procedure rates and clinical outcome rates). As I have explained in scores of places – including the Scanlan’s Rule page (SR) – such increases will tend toward causing the following changes in measures of differences between rates:
(1) Relative differences in experiencing those favorable outcomes will tend to decrease.
(2) Relative differences in failing to experience those favorable outcomes will tend to increase.
(3) Absolute differences in fairly uncommon outcomes will tend to increase while absolute differences in fairly common outcomes will tend to decline. More specifically, as outcome rates move toward a range defined by a rate of 50% for each group, absolute differences will tend to increase; as outcomes move away from that range, absolute differences will tend to decrease. Within the range, the statistically driven pattern is more complicated, though the variation within the range will tend to be small. See Introductory Section of the Scanlan’s Rule page.
(4) Odds ratios will tend to change in the opposite direction of absolute differences.
Thus, those researchers who measure disparities in terms of relative differences in favorable outcomes – and this remains a common approach notwithstanding the recommendation of the National Center for Health Statistics (NCHS) that all disparities be measured in terms of relative differences in adverse outcomes – will tend to find disparities to decline.
NCHS, which measures all disparities in terms of relative differences in adverse outcomes, will tend to find disparities to increase.
The Agency for Healthcare Research and Quality (AHRQ), which measures disparities in terms of the larger of the two relative differences (in the favorable or the adverse outcome), will tend to find disparities in relatively uncommon favorable outcomes to increase and disparities in relative common favorable outcomes to decrease – as explained on SR. In January 2011, the Centers for Disease Control and Prevention (CDC) issued a report on health and healthcare disparities in which it principally relied on absolute differences between rates. Since absolute differences tend to change in the opposite direction of the larger relative difference as the prevalence of an outcome changes, CDC will tend to reach conclusions as to the comparative size of disparities that are the opposite of those AHRQ would reach.
For extensive critiques of the NCHS and AHRQ, and CDC approaches see Section E.4 of MHD and A.6 of SR and the references noted in those sections (especially references 6 and 7 below).
Like the CDC, many private sector researchers measure healthcare disparities in terms of absolute differences between rates. This is the usual (though not invariable) approach of the Health Policy Group of Harvard Medical School and, perhaps due to the influence of that group, the approach in the Massachusetts P4P program. Like the CDC, such researchers will thus tend to find disparities in fairly uncommon outcomes to increase, but find disparities in fairly common outcomes to decline.
Researchers relying on odds ratios will tend to find disparities to change in the opposite direction of that found by researcher who rely on absolute differences. Researchers who rely on absolute differences, but then employ logistic regression and the odds ratios generated thereby to determine whether results were affected by confounders, may find the directions of changes to be the opposite in the adjusted analysis from that in the original analysis. See Section 1 of the Adjustment Issues sub-page of the Vignettes page.
An additional complication with regard to measures of disparities in clinical outcomes involves the fact that clinical outcome rates may be determined on the basis of (a) the rates of advantaged and disadvantage groups within the overall population or (b) the rates of advantaged and disadvantaged groups within subpopulations warranting special attention. The latter often will be a truncated portion of the former, and, being such, will not involve normal distributions of risks of the two groups even when the distributions in the larger populations are perfectly normal.
Within the truncated population, the distributions will tend to exhibit the same patterns of changes in relative differences and absolute differences (though not of odds ratios) as observed in the larger population. Nevertheless, there will be an important implication with regard to measures of absolute differences. For, while the favorable outcome may be quite common within the overall population, it may be fairly uncommon within the truncated population. For purposes of a simple illustration, let us define hypertension solely in terms of systolic blood pressure above 139. If improvements in care were to enable everyone with systolic blood pressure (SBP) between 140 and 149 to reduce their SBP below 140, absolute differences between black and white hypertension rates within the population at large (or a particular entity’s patients) would tend to decline. But the racial disparity in control of hypertension within a population deemed hypertensive on the basis an initial SBP level above 139 would tend to increase.
The reductions in SBP would tend to have the same effect on relative differences within both the larger population and the truncated population. That is, relative differences in hypertension rates would tend to increase while relative differences in absence of hypertension would tend to decline. Thus, for example, NCHS would tend to find disparities to increase in both the overall population and the population deemed hypertensive.
Even though the relative differences would tend to change in the same direction for both populations, however, which population is examined could have an impact on AHRQ’s determination of the direction of change in the disparity. For, within the larger population, the disparity in the adverse outcome would typically be larger than the disparity in the favorable outcome. Hence, examining disparities within that population, AHRQ, like NCHS, would rely on relative differences in the adverse outcome and tend to regard the disparity as increasing. But within the truncated population, the relative difference in the favorable outcome could well be larger than the relative difference in the adverse outcome. Hence, relying on the relative difference in the favorable outcome, AHRQ would tend to reach a different conclusion from NCHS regarding the direction of change in disparity.
These patterns are illustrated in Figures 4, 7-12 of reference 8 (ICHPS 2008). Due to the small numbers of blacks and consequent irregularities in the National Health and Nutrition Survey SBP data, the patterns in Figure 10 are not precisely in accord with the standard patterns of relative differences. But the concept is nevertheless satisfactorily illustrated. As with all discussion of the comparative of size of relative differences in experiencing and avoiding an outcome, the points made on the Semantic Issues sub-page sub-page of SR must be borne in mind. For discussion of actual patterns of changes in absolute differences in larger populations and the subpopulations deemed in need of control, see reference 9. Such reference shows how the view that it is easier to reduce disparities in process outcomes than in clinical outcomes, while probably correct, is largely based on a failure to recognize the expected patterns of changes in absolute differences between rates for each type of outcome.
A further issue with regard to disparities in control among subpopulations involves the fact that the approach described on the Solutions sub-page of MHD for measuring changes in disparities that is unaffected by the prevalence of an outcome – which is probably insufficiently precise for analyses of disparities within a larger population to be used in a P4P program – is not even theoretically sound in the case of a truncated distribution (as discussed with regard to Tables 7 and 8 in reference 10).
In sum, so far there is not a basis for concluding what impact P4P will likely have healthcare disparities. But, until there exits a more widespread understanding of the effect of overall prevalence on each measure of differences between rates than currently exists, and until there exists a better means of identifying meaningful changes in healthcare disparities than has so far been developed, it would certainly be a mistake to attempt to include effects on disparities as performance element in any P4P program.
See also the Truncation Issues subpage of the Scanlan’s Rule page.
Addendum: As a result of continuing attention to P4P and healthcare disparities, I have posted a number of additional on-line comments. They are listed as reference 11-16 below. Item 11 warrants note because the subject article plays a role in conflicting perceptions in the United States and the United Kingdom as to the effects of P4P on healthcare disparities. In the United States, based on the use of absolute differences to measure disparities in an uncommon outcome (where increases in outcome rates tend to increase absolute differences between rates), there exists a perception that P4P will increase disparities; in the United Kingdom, based on the use of absolute differences to measure disparities in common outcomes (where increases in outcome rates tend to reduce absolute differences between rates), there exists a perception that P4P will tend to reduce disparities.
References:
1. Scanlan JP. Changing social inequalities in SIDS. Am J Public Health Dec. 11, 2005 (responding to Pickett et al. Widening social inequalities in risk for sudden infant death syndrome. Am J Public Health 2005;95:97-81: http://jpscanlan.com/images/Comment_on_Pickett_et_al._AJPH_2005.pdf
2. Scanlan JP. Pay-for-performance implications of the failure to recognize the way changes in prevalence of an outcome affect measures of racial disparities in experiencing the outcome. Journal Review Feb. 8, 2008 (responding to Werner, RM, Asch DA, Polsky D. Racial profiling: The unintended consequences of coronary artery bypass graft report cards. Circulation 2005;111:1257–63): http://journalreview.org/v2/articles/view/15769766.html
3. Scanlan JP. First learn to measure healthcare disparities. Health Affairs Mar. 12, 2008 (responding to Casalino LP, Elster A, Eisenberg A, et al. Will pay-for-performance and quality reporting affect health care disparities? Health Affairs 2007;26(3):405-414):: http://content.healthaffairs.org/cgi/eletters/26/3/w405
4. Scanlan JP. Inclusion of healthcare disparities issues in pay-for-performance programs should await development of reliable means of measuring changes in disparities over time. Journal Review Feb. 16, 2008 (responding to Casalino LP, Elster A, Eisenberg A, et al. Will pay-for-performance and quality reporting affect health care disparities? Health Affairs 2007;26(3):405-414): http://journalreview.org/v2/articles/view/17426053.html
5. Scanlan JP. Pay-for-performance and the measurement of healthcare disparities. Journal Review Feb. 10, 2008 (responding to Chien AT, Chin MH, Davis AM, Casalino LP. Pay for performance, public reporting, and racial disparities in health car: how are programs being designed. Med Car Res Rev 2007;64:283S-304S)
http://jpscanlan.com/images/Chien_MCRR_2007.pdf
6. Scanlan JP. Measurement Problems in the National Healthcare Disparities Report, presented at American Public Health Association 135th Annual Meeting & Exposition, Washington, DC, Nov. 3-7, 2007.
PowerPoint Presentation: http://www.jpscanlan.com/images/APHA_2007_Presentation.ppt
Oral Presentation: http://www.jpscanlan.com/images/ORAL_ANNOTATED.pdf
Addendum (March 11, 2008): http://www.jpscanlan.com/images/Addendum.pdf
Abstract: http://apha.confex.com/apha/135am/techprogram/paper_153201.htm
7. Scanlan JP. Study illustrates ways in which the direction of a change in disparity turns on the measure chosen. Pediatrics Mar. 27, 2008 (responding to Morita JY, Ramirez E, Trick WE. Effect of school-entry vaccination requirements on racial and ethnic disparities in Hepatitis B immunization coverage among public high school students. Pediatrics 2008;121:e547-e552): http://pediatrics.aappublications.org/cgi/eletters/121/3/e547
8. Scanlan JP. Can We Actually Measure Health Disparities?, presented at the 7th International Conference on Health Policy Statistics, Philadelphia, PA, Jan. 17-18, 2008 (invited session).
Abstract:http://www.amstat.org/meetings/ichps/2008/index.cfm?fuseaction=AbstractDetails&AbstractID=300283
PowerPoint Presentation: http://www.jpscanlan.com/images/2008_ICHPS.ppt
Oral Presentation: http://www.jpscanlan.com/images/2008_ICHPS_Oral.pdf
9. Scanlan JP. Understanding patterns of correlations between plan quality and different measures of healthcare disparities. Journal Review Aug. 30, 2007 (responding to Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Relationship between quality of care and racial disparities in Medicare health plans. JAMA 2006;296:1998-2004):
http://jpscanlan.com/images/Trivedi_JAMA_2006.pdf
http://jpscanlan.com/images/Trivedi_JAMA_2006_Correction.pdf
http://jpscanlan.com/images/Trivedi_JAMA_2006_Follow.pdf
10. Scanlan JP. Comparing the size of inequalities in dichotomous measures in light of the standard correlations between such measures and the prevalence of an outcome. Journal Review Jan. 14, 2008 (responding to Boström G, Rosén M. Measuring social inequalities in health – politics or science? Scan J Public Health 2003;31:211-215):
http://www.jpscanlan.com/images/Bostrom_and_Rosen_Comment.pdf
11. Scanlan JP. Interpreting patterns of changes in absolute differences between rates when common outcomes become even more common. BMJ Dec. 7, 2008 (responding to Ashworth M, Medina J, Morgan M. Effect of social deprivation on blood pressure monitoring and control in England: a survey of data from the quality and outcomes framework. BMJ 2008;337:a2030): http://www.bmj.com/cgi/eletters/337/oct28_2/a2030
12. Scanlan JP. Measuring racial disparities in hypertension control. Ann Fam Med Jan. 25, 2009 (responding to Satcher D. Examining racial and ethnic disparities in health and hypertension control. Ann Fam Med 2008;6:483-485): http://www.annfammed.org/cgi/eletters/6/6/483
13. Scanlan JP. Tying pay-for-performance to healthcare disparities should await mastery of measurement issues. BMJ Feb. 8, 2009 (responding to Bierman AS, Clark JP. Performance measure and equity. BMJ 2007;334:1333-1334): http://www.bmj.com/cgi/eletters/334/7608/1333
14. Scanlan JP. Recommendations to incorporate reductions in disparities in P4P programs cannot ignore measurement issues. Journal Review Feb. 21, 2009 (responding to Chien AT, Chin MH. Incorporating disparity reduction into pay-for-performance. J Gen Intern Med 2008;24(1):135-136): http://jpscanlan.com/images/Chien_JGIM_2008.pdf
15. Scanlan JP. Authors recommending financial incentives to reduce disparities must confront measurement issues. Pediatrics Jan. 9, 2010 (Chin MH, Alexander-Young M, Burnet DJ. Health care quality-improvement approaches to reducing child health disparities. Pediatrics 2009;14:S224-S236):http://www.pediatrics.org/cgi/content/full/124/Supplement_3/S224)
16. Scanlan JP. Incentive programs to reduce healthcare disparities should await better understanding of how to measure those disparities. Journal Review March 2, 2010 (responding to Siegel B, Nolan L. Leveling the field – ensuring equity through National Health Care Reform. N Engl J Med 2009;361:2401-2403):
http://jpscanlan.com/images/Siegel_NEJM_2009.pdf
http://journalreview.org/v2/articles/view/19955518.html
17. Perverse Perceptions of the Impact of Pay for Performance on Healthcare Disparities, presented at the 9th International Conferences on Health Policy Statistics, Cleveland, Ohio, Oct. 5-7, 2011:
Abstract: http://www.amstat.org/meetings/ichps/2011/index.cfm?fuseaction=AbstractDetails&AbstractID=301436
PowerPoint: http://jpscanlan.com/images/Scanlan_ICHPS_2011.ppt
Oral: http://jpscanlan.com/images/ICHPS_2011_Oral.pdf