Records

Furberg CD (2009). How should one analyse and interpret clinical trials in which patients don’t take the treatments assigned to them?

Commentary on: Coronary Drug Project Research Group (1980). Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. N Engl J Med 303:1038-41.

Cite as: Furberg CD (2009). How should one analyse and interpret clinical trials in which patients don’t take the treatments assigned to them? The James Lind Library(www.jameslindlibrary.org).
Accessed © Curt D Furberg, 2009.

Author contact details: Curt D Furberg, Division of Public Health Sciences, Wake Forest University School of Medicine, Winston-Salem, NC 27157, USA.

Two main ways to analyze clinical trial data
There are two main ways to analyze clinical trial data, and these have been recognised since the beginning of the modern era of clinical trials. In 1941, in his report of a controlled trial of a whooping cough vaccine, Joseph Bell wrote as follows:

“Obviously it is not practically possible to preselect two large strictly random groups of children who are representative of the general population and to insure that every child in one group receives the vaccine while every child in the other group receives no vaccine during the observation period. Children in the general population have the prerogative to refuse vaccine offered and the liberty to obtain other vaccine when desired. In these premises there is no known way of changing the two groups so that one would include only children actually vaccinated, and the other include only children not vaccinated, without destroying the randomness of the selection and to that extent possibly invalidating the answer to the question asked. After it has been established that the vaccine confers protection, then questions concerning the amount and duration of such protection might in part demand direct comparison of the experience of the children actually vaccinated with those not vaccinated, providing adequate data are at hand to equalize the two groups with respect to attributes which apparently influence the occurrence of the disease.
“For this report, the approach to the primary problem involved the preselection of two large strictly random groups of children and the subsequent injection of a large proportion of only one group with the vaccine. All analyses herein presented are a comparison of the experience of such preselected groups regardless of their actual status with respect to receiving the vaccine. The difficulties encountered in this approach are chronologically described in detail so that the reader may evaluate any possible errors involved.” (Bell 1941)

The primary analysis used by Joseph Bell thus required that all randomized participants be included in the analysis according to their original study group allocation. This is called the intention-to-treat approach. It retains the principal benefit of randomization, namely that comparison groups remain comparable with respect to known and unknown risk factors.

Joseph Bell also refers to the alternative approach to analysis, however. This involves direct comparison of the experience of the children actually vaccinated with those not vaccinated. This is sometime referred to as “analysis by treatment administered.” Another term is “per-protocol”, which is a misnomer and should be avoided. The argument against the intention-to-treat approach is that certain randomized participants should not be included in the analysis, for example, those later found to be ineligible, those not taking their study medication as stated in the protocol, and those with poor or missing data. Reasonable people might well observe that including such people in the analysis will tend to underestimate any favorable (and unfavorable) treatment effects – participants who don’t receive the allocated intervention as prescribed can’t benefit or suffer from it. Additionally, patients may want to know what will happen if they are good adherers. The drawback of this approach to the analysis is that it erodes the advantage of randomization. Bell recognised this when he referred to the need for adequate data “to equalize the two groups with respect to attributes which apparently influence the occurrence of the disease.” Unfortunately such ‘statistical equalization’ can only be pursued using measured factors of potential prognostic importance. There is no way of taking account – as random allocation does – of unmeasured factors.

The critical question is – to whom should the adherent participants in the intervention group(s) be compared? It is obviously important that the study subgroups are comparable in terms of prognosis or risk – that like will be compared with like. Unfortunately, one cannot assume that adherent participants in one group are always comparable to adherent participants in another group. After all, many participants receiving active treatment withdraw due to adverse effects, while placebo participants may withdraw due to perceived lack of benefit. In addition, the proportion of non-adherers in the study groups may be very different. Will withdrawing potentially different subsets of participants from the study groups invalidate the comparability achieved by random allocation?

A landmark study
The Coronary Drug Project evaluated several lipid-modifying drugs, including clofibrate, in survivors of a myocardial infarction (Coronary Drug Project Research Group 1980). The overall 5-year mortality in the clofibrate group was 18.2% compared to 19.4% in the placebo group, a statistically non-significant difference. Among patients allocated to clofibrate who took 80% of the study medication the mortality was 15.0%. This was strikingly lower than the mortality among those patients with lower adherence, which was 24.6%. One possible explanation for this difference is that clofibrate did, after all, have an impressive beneficial mortality effect when post-infarction patients actually took the drug: the proponents of the per protocol approach to analysis appeared to have a case. However, the level of medication adherence was similar among those allocated placebo as it was in the clofibrate group, so an analysis was done to compare mortality among adherers and non-adherers to placebo. This revealed a difference in 5-year mortality mirroring that for adherers and non-adherers to clofibrate: placebo adherers had a mortality of 15.1%, compared with 28.2% among poor adherers. Importantly, attempts at ‘equalizing’ these non-randomized subgroup comparisons using 40 baseline characteristics associated with 5-year mortality had only a small effect on the observed differences.

What inferences can be drawn from these analyses? First, they demonstrated that participants with high and low adherence in both comparison groups were different in terms of 5-year mortality risk. Good adherers are very different from low adherers. Second, the similarity in mortality and adherence rates across the study groups suggests that, in this population, any effects of clofibrate were,on average, similar to placebo.

The next questions are – Have these remarkable findings been confirmed in other studies? Replications are important in research. What are the causes of this adherence effect?

A confirmatory study
A similar pattern to that observed in the Coronary Drug Project was observed in the Aspirin Myocardial Infarction Study (Friedman et al. 1998). Overall, no statistically significant difference in mortality was observed between patients assigned to aspirin and those assigned to placebo (10.9% vs. 9.7%). The level of medication adherence was high in both groups. However, participants with good adherence had mortality rates of 6.1% and 5.1%, respectively, and those with low adherence rates of 21.9% and 22.0%, respectively. Thus, in a similar population, there were marked mortality differences between participants with high and low adherence rates and this difference was observed in both the aspirin and placebo groups. These analyses of the Aspirin Myocardial Infarction Study thus confirmed the analyses of the Coronary Drug Project. The next questions are – Are the analyses of these two studies reflected in other patient populations receiving interventions with documented mortality effects?

Other observations
Published findings from other trials in treatments of myocardial infarction, congestive heart failure, arrhythmias, cancer and schizophrenia have also confirmed the findings from the Coronary Drug Project (Czajkowski et al. 2009), and a recent analysis of observational data found that better adherence to statin treatment was associated with lower mortality (Rasmussen et al. 2007). The association between good adherence and favorable health outcomes was confirmed in a recent meta-analysis (which unaccountably failed to include the findings from the Coronary Drug Project and the Aspirin Myocardial Infarction Study) (Simpson et al. 2006). The overall odds ratios were 0.56 (95% CI 0.43-0.74) for adherence to placebo and 0.55 (0.49 – 0.62) for adherence to drug therapy. The authors speculate that good medication adherence may be “a surrogate marker for overall healthy behavior.” Interestingly, the meta-analysis also reported that good adherence to harmful drugs was associated with increased mortality (odds ratio 2.90, 1.04-8.11).

Lack of any association between adherence and outcomes has also been observed or reported. The findings for the clofibrate group in the Coronary Drug Project was not confirmed in its high-dose estrogen group (unpublished data). A possible explanation for this difference might be that estrogen had serious adverse effects and that there were twice as many non-adherers in the estrogen group as in the placebo group. Two published analyses that failed to find any relationship between medication adherence and health outcomes differed from those mentioned above in two regards (Czajkowski et al. 2009). First, the treatment outcome was not all-cause mortality; second, they were analyzed using adherence as a continuous variable. One of them was the Cardiac Arrhythmia Suppression Trial, which was stopped early due to harm (Oblas-Manno et al. 1996). The other was the Lipid Research Clinics – Coronary Primary Prevention Trial. The primary outcome included fatal and nonfatal myocardial infarction (Lipid Research Clinics Program 1984).

The picture is clearly complex and various explanations have been offered for the observations outlined in this commentary. The ‘clofibrate findings’ could reflect a ‘healthy adherer’ effect, with good adherence being a marker of better health or a healthier lifestyle. Sicker patients may not tolerate a study medication and stop taking it. This could explain a difference in the active-treated group but not among placebo participants. Another explanation could be that patients develop medical conditions or serious complications that could lead to low adherence as well as worse prognosis.

What lessons have we learned so far?

• Good and poor medication adherers seem to have different prognoses.
• Good adherence to harmful drugs is associated with a worse prognosis.
• Good adherence to beneficial drugs is associated with a better prognosis.
• Specific reasons that could account for the relationship between good adherers and favorable outcomes and poor adherers and unfavorable outcomes remain unclear.
• Thus, there is no established method to adjust for adherence-related participant factors.
• There is no guarantee that subsets of participants with high or low compliance within two study groups are comparable in terms of risk.
• Analysis of clinical trial data by treatment administered can be misleading.
• The intention-to-treat approach to analysis remains the safest or least biased way of analyzing clinical trial data.
• This is the reason why reputable medical journals and regulatory agencies adhere to the intention-to-treat approach.

This James Lind Library commentary has been republished in the Journal of the Royal Society of Medicine 2010;103:202-204.

References

Bell JA (1941). Pertussis prophylaxis with two doses of alum-precipitated vaccine. Public Health Reports 56:1535-1546.

Coronary Drug Project Research Group (1980). Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. N Engl J Med 303:1038-41.

Czajkowski SM, Chesney MA, Smith AW (2009). Adherence and placebo effect. In: Shumaker SA, Ockene JK, Riekert KA, eds. The Handbook of Health Behavior Change. Third Edition. New York: Springer Publishing Company.

Friedman LM, Furberg CD, DeMets DL (1998). Fundamentals of clinical trials, 3rd ed. St. Louis: Mosby, 1996; New York: Springer-Verlag, 1998.

Lipid Research Clinics Program (1984). The Lipid Research Clinics Coronary Primary Prevention Trial results II: The relationship of reduction in incidence of coronary heart disease to cholesterol lowering. JAMA 251:365-374.

Oblas-Manno D, Friedmann E, Brooks MM, Thomas SA, Haakenson C, Morris M, et al. (1996). Adherence and arrhythmic mortality in the Cardiac Arrhythmia Suppression Trial (CAST). Ann Epidemiol 6:93-101.

Rasmussen JN, Chong A, Alter DA (2007). Relationship between adherence to evidence-based pharmacotherapy and long-term mortality after acute myocardial infarction. JAMA 297:177-86.

Simpson SH, Eurich DT, Majumdar SR, Padwal RS, Tsuyuki RT, Varney J, Johnson JA (2006). A meta-analysis of the association between adherence to drug therapy and mortality. BMJ 333:15-19. doi:10.1136/bmj.38875.675486.55.