INEVITABILITY OF INTERACTION
(draft, Dec. 24, 2011; rev. Mar. 8, 2012)
This page is related to Subgroup Effects sub-page (especially Section D) and the Illogical Premises and Illogical Premises II sub-pages of the Scanlan’s Rule page of jpscanlan.com. The Illogical Premises sub-page and the referenced section of the Subgroup Effects sub-page involve the fact that it is illogical to expect that a factor will typically cause the same proportional changes in different baseline rates given that it is impossible for a factor to cause equal proportionate changes in different rates of experiencing an outcome while at the same time causing equal proportionate changes in the opposite outcome. As put in the Illogical Premises page:
But the premise that, absent the occurrence of something meaningful, a factor will cause equal proportionate changes in two different base rates is an illogical one for the simple reason that a factor cannot cause equal proportionate changes in two different base rates of experiencing an outcome while at the same time causing equal proportionate changes in the rates of avoiding the outcome (i.e., experiencing the opposite outcome). That is, if Group A has a base rate of 5% and Group B has a base rate of 10%, a factor that reduces the two rates by equal proportionate amounts, say 20% (from 5% to 4% and from 10% to 8%) would necessarily increase the opposite outcome by two different proportionate amounts (95% increased to 96%, a 1.05% increase; 90% to 92%, a 2.2% increase). And since there is no more reason to expect that two group would undergo equal proportionate changes in one outcome than there is to expect them to undergo equal proportionate change in the opposite outcome, there is no reason to regard it as somehow normal that the two groups would undergo the same proportionate change in either outcome.
The Subgroup Effects page addresses the reasons to expect that a factor that similarly affects two groups with different baseline rates of experiencing an outcome will tend to show a larger relative effect in the group with the lower baseline rate while showing a larger relative effect in the opposite outcome for the other group. In other words, a factor that reduces mortality will tend to cause a larger relative reduction in mortality in the group with the lower baseline mortality rate while causing a larger relative increase in survival in the other group.
This page addresses the matter from a slightly different perspective. Specifically, according to the standard approach of identifying interaction on the basis on the absence of equal proportionate changes in a baseline, when two groups have different baseline rates, there necessarily will be an interaction either as to one outcome or as to the other. Subsequent to the initial creation of this page, I created the Illogical Premises II sub-page, which discusses, in terms of a challenge to the rate ratio as a measure of association, that it is illogical to regard a particular rate ratio as reflecting the same strength of association as to different baseline rates (e.g., a rate ratio of 2.0 with respect both to a rate pair 30% and 15% and a rate pair of 20% and 10%), given that if the rate ratio for one outcome is the same the rate ratio for the opposite outcome necessarily will be different. Thus, the Illogical Premises II sub-page is closely related to the instant subject, though not in the context of subgroup analysis.
For the present, apart from the above, I limit the discussion of the inevitability of interaction to that set out in a December 19, 2011 comment on a 2003 BMJ Statistics Note (Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ 2003;326:219), which comment may be found here, and which is set out below:
The Inevitability of Interaction
This follows on a September 2010 comment [1] on the Altman and Bland Statistics Note [2] that provided a formula for calculating the likelihood that a seeming interaction – also termed effect heterogeneity or subgroup effect – occurred by chance. The purpose of this comment is to show why the very concept of interaction, as commonly understood, is illogical, showing as well why such interaction is inevitable.
The premise underlying the formula presented by Altman and Bland (which formula can now be applied with an online calculator [3]), is that, absent interaction, an intervention will show the same relative risk (RR) for two groups even when the groups’ baseline rates are different. That is, if an intervention reduces a baseline rate from 20% to 10% (RR=.5) for Group A, absent interaction, one would expect the intervention to reduce Group B’s baseline rate of 10% to 5%. Tests such as that presented by Altman are aimed at determining, for example, where the 10% rate is instead reduced to 3%, how likely it is that such reduction (RR=.3) would occur by chance if the relative risks for the two groups are in fact the same.
But assume that in fact we observe the same .5 relative risk in both situations. There would be no question of interaction and, regardless of the confidence intervals that would go into the Altman/Bland formula, the z-score would be 0.
If that occurred, however, the relative risks for the opposite outcome would be different for the two groups. Group A’s relative risk would be 1.125 (80% increased to 90%) and Group B’s would be 1.055 (90% increased to 95%). This holds with any pair of baseline rates. That is, it is not possible for different baseline rates to change by equal proportionate amounts unless the opposite outcomes change by different proportionate amounts. Since there is no more reason to expect that the two groups will experience equal proportionate changes in one outcome than in the opposite outcome, it is illogical to think it somehow normal that that they will experience equal proportionate changes in either outcome.
Further, features of normal distributions of risk provide reason to expect that a factor that similarly affects two groups with different baseline rates for an outcome will effect a larger proportionate change for the group with the lower baseline rate while effecting a larger proportionate change in the opposite outcome for the other group.[4-6] Notably, the example used by Altman and Bland, from a meta-analysis by Torgerson and Bell-Syer [7] of the effect of hormone replacement therapy on nonvertebral fractures among women, showed a larger relative risk reduction in women under age 60 (RR=.67) than women over age 59 (RR=.88). Given that younger women would have lower baseline rates, the pattern conforms to the expectations based on the distributional forces just described.
Proceeding from a perspective where the relative risks are assumed to be equal absent a sound showing that they are not, Altman and Bland, based on the calculation of a z-score of 1.24 ( p=0.2), concluded that “[t]here is no good evidence to support a different treatment effect of younger and older women.” But the absence of any reason to expect (indeed, given that it is illogical to expect) that relative treatment effects would be the same for different baseline rates, and sound reason to believe that the relative effect would be larger for the group with the lower baseline rate, there is compelling reason to believe that the relative effect in fact is larger for younger women.
The practical implications of assumptions about effects across different baseline rates seem greatest when it is necessary to estimate, on the basis of a risk reduction observed as to one baseline rate, the clinically crucial absolute risk reduction for other baseline rates. The most plausible approach to that problem is to derive from the observed rates for treatment and control groups the difference between means of the hypothesized underlying distributions and use that difference to estimate absolute risk reductions for the baseline rates of concern. Results of such approach are illustrated in Table 3 (Method 1) of reference 8, which also shows the varying absolute risk reductions that would be estimated on the basis of assumptions of equal proportionate changes in one outcome (Method 2) or equal proportionate changes in the opposite outcome (Method 2 alt).
The assumption underlying Method 1 is that an intervention will shift the risk distributions of each group an equivalent amount. While the applicability of the assumption to a particular setting may be questioned, it at least has a rational basis. The assumption of equal proportionate risk reductions does not. Indeed, it is fairer to assume that the reductions always will be different (save on the rare occasion when a meaningful interaction, by happenstance, causes them to coincide) than that they typically will be the same.
See the penultimate paragraph of reference 9 regarding an assumption of proportionate changes in odds ratios. That item also explains why it is essential that studies present underlying rates, a point implicit in the discussion above.
References:
1. Scanlan JP. Problems in identifying interaction where groups have different base rates. BMJ 21 Sept 2010: http://www.bmj.com/content/326/7382/219/reply#bmj_el_241943
2. Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ 2003;326:219.
3. http://www.hutchon.net/CompareRR.htm
4. Scanlan JP. Divining difference. Chance 1994;7(4):38-9,48: http://jpscanlan.com/images/Divining_Difference.pdf
5. Scanlan JP. Race and mortality. Society 2000;37(2):19-35: http://www.jpscanlan.com/images/Race_and_Mortality.pdf
6. Interpreting Differential Effects in Light of Fundamental Statistical Tendencies, presented at 2009 Joint Statistical Meetings of the American Statistical Association, International Biometric Society, Institute for Mathematical Statistics, and Canadian Statistical Society, Washington, DC, Aug. 1-6, 2009: http://www.jpscanlan.com/images/JSM_2009_ORAL.pdf
7. Torgerson DJ, Bell-Syer SEM. Hormone replacement therapy and prevention of nonvertebral fractures. JAMA 2001;285:2891-7.
8. Subgroup Effects sub-page of Scanlan’s Rule page of jpscanlan.com:http://www.jpscanlan.com/scanlansrule/subgroupeffects.html
9. Ratio measures are not transportable. BMJ Nov. 11, 2011 (responding to Schwartz LS, Woloshin S, Dvorin EL, Welch HG. Ratio measures in leading medical journals: structured review of underlying absolute risks. BMJ 2006;333:1248-1252): http://www.bmj.com/content/333/7581/1248?tab=responses
|