Introduction
The purpose of this section is to appraise the non-inferiority margin (NIM) used in the EINSTEIN trials. Both EINSTEIN trials were designed to test whether rivaroxaban was non-inferior to the standard of care in reducing the recurrence of VTE events.8 The standard of care was considered to be heparin for the initial period post-VTE event, administered concomitantly with VKA, and the VKA was continued alone once its therapeutic range was attained. The NIM for the hazard ratio, in the EINSTEIN trials, was derived from historical trials controlled by placebo, no treatment, or standard of care.5,8 The indirect confidence interval comparison method was used. This method estimated the NIM from the upper limit of the 95% confidence interval of the odds ratio for the standard of care versus placebo or no treatment, and the upper limit of the 95% confidence interval of the hazard ratio for rivaroxaban versus the standard of care for the statistical test.
Non-Inferiority Margin Estimates by Measure of Effect, Scale, and Percentage of Pooled Effect Preserved.
Summary of the Historical Trials and Constancy Assumption
A meta-analysis published by Prins and Lensing summarized the historical trials used for the NIM estimation.8 The characteristics of the included studies are presented in . The trials included patients with PE only (two trials),46,47 DVT only (seven trials)32,34,48–52, and both PE and DVT (five trials).35–37,53,54 The EINSTEIN trials included proximal DVT patients only; however, the meta-analysis included five trials in which patients with distal DVT were also included.32–36 Another trial included patients with unstable angina or arterial ischemia along with VTE patients.37 The management and outcomes of these patients might not reflect the proximal DVT patients included in the EINSTEIN trials. Although arguments can be made to include or exclude non-proximal DVT events in calculating the NIM, it was not a major determinant of the NIM in the ENSTIEN trials.
Summary of the CADTH Common Drug Review Recalculated Non-Inferiority Margins.
Additionally, the intervention and comparator groups used in this meta-analysis were not those typically used to estimate the NIM; estimation of the NIM is usually based on the comparison between the comparator used in the new trial and placebo.55,56 For the EINSTEIN trials, a meta-analysis of randomized controlled trials (RCTs) comparing anticoagulation with unfractionated heparin plus VKA with placebo would be the most appropriate to estimate the NIM. The meta-analysis by Prins and Lensing identified two placebo-controlled trials;46,57 however, these studies were > 25 years old and used anticoagulant therapy for short duration (14 days or three months only). For these reasons, the authors judged that the NIM could not be estimated from these trials alone. Instead, the meta-analysis also included RCTs that compared different anticoagulation interventions. These interventions were grouped as “more effective” and “less effective” treatments. The meta-analysis did not provide a specific definition for the “more/less effective” groups; these included different durations of anticoagulants, different heparin formulations, and placebo or no treatment.
Finally, the outcome in the included studies was recurrence of VTE events; however, mortality was not specified as part of this outcome. The odds ratios consistently favoured the “more effective” treatment, except in one trial that compared the use of VKA for one week with 12 weeks. This trial showed that the rate of recurrence, within six months, was higher in the longer-duration treatment group, but the difference was not statistically significant.8,54
Calculation Method
The meta-analysis by Prins and Lensing estimated the odds ratio and the relative risk, using both arithmetic and geometric scales.8 The odds ratio was calculated using the Mantel–Haenszel procedure, which assumed a fixed treatment effect. Our observation is that there are two schools of thought related to using fixed or random effects models. Some would advocate evaluating heterogeneity and then determining whether a fixed or random effects model is more appropriate. Others would advocate using only the more conservative approach using a random effects model. The modelling approach in determining the NIM appears to suggest the random effects model.58,59 Schumi and Wittes note in their article:60 “The FDA Guidance suggests a preference for so-called ‘random-effects models’ in meta-analysis that will be used to establish the margin in non-inferiority trials. These models in contrast to MH [Mantel–Haenszel] and Peto approaches, make very specific assumptions about the distribution of the effect sizes across all potential studies.” In fact, the Prins and Lensing meta-analysis grouped different regimens and durations of anticoagulation interventions into the “more-effective” and “less-effective” groups; these interventions produced upper limits of the 95% confidence interval for the odds ratios that ranged from 0.0 to 17.9.8
Based on methods used in the meta-analysis, the odds ratio was 0.18 (95% CI, 0.14 to 0.25) and the relative risk was 0.19 (95% CI, 0.12 to 0.28). The NIM was estimated from these effect sizes using the following formula:
AND
The associated NIMs were reported as follows:8
The EINSTEIN trials used the highest NIM estimated (2.00), but the trial protocol did not provide justifications for the choice of odds ratio over risk ratio or the arithmetic scale over the geometric scale for the NIM estimation. Prins and Lensing reported that the geometric scale would be the logical choice when evaluating event rates in a “no-treatment” group, but the arithmetic or linear scale was used instead because the event rates were compared with the standard of care.8
CADTH Common Drug Review Re-assessment of the Non-inferiority Margin
Based on the aforementioned observations on the calculation method and the data discrepancies, CDR reviewers recalculated the NIM using the recurrence rates reported by Prins and Lensing, based on fixed and random treatment effects model for the odds ratio and relative risk. Random effects models were chosen for the CDR calculations, given the clinical heterogeneity between the studies included in the Prins and Lensing meta-analysis; hence, a common effect size could not be assumed.58,59,61 Another methodological uncertainty is that the NIM estimation was based on the arithmetic scale rather than the geometric scale estimations, despite both being calculated by Prins and Lensing. The use of arithmetic scale induces some bias when the sample size is small and the prevalence of the outcome is low.62 The rationale behind using the geometric scale when conducting a meta-analysis is to overcome the issue of skewed data. This was illustrated in sections 7.7.3.4 and 9.4.5.3 of the Cochrane handbook; however, the handbook used the example of continuous data without specifying the case for dichotomous variables.63 The paper by Ukoumunne et al. confirms that the geometric scale is may be used on binary outcomes.62 Although the paper was about cluster RCTs rather than meta-analyses, it provided additional evidence that the geometric scale should be used whenever skewness of the data is suspected. Furthermore, two key papers on the statistical derivation of NIMs demonstrated that margins can be estimated using the logarithmic form of the effect size (i.e., on the geometric scale).64,65 In his paper, Wang reported that the evaluation of non-inferiority is made while the effect size and the NIM are on the logarithmic scale (i.e., the geometric form); he reported “if the upper 100(1-γ)% confidence limit for the risk difference log(Treatment) - log(Control) is less than the margin log(NIM) in the active controlled trial, then it can be concluded that the new treatment is not inferior to the active control in the sense of preserving the specified fraction of the control effect.”65 It was not clear from the Prins and Lensing article why the arithmetic scale was used instead of the geometric scale in estimating the NIM. For the CDR re-assessment, NIMs were estimated using the arithmetic and geometric scales. The resulting odds ratio, relative risk, and the associated NIMs are summarized in .
The recalculated NIMs ranged from 1.87 to 2.21 on the arithmetic scale. The hazard ratios for VTE recurrence in the EINSTEIN trials were 0.68 (95% CI, 0.44 to 1.04) in the DVT trial and 1.12 (95% CI, 0.75 to 1.68) in the PE trial; therefore, the use of arithmetic scale estimation did not affect the non-inferiority conclusion. However, the geometric scale estimation produced NIMs of 1.54 to 1.67 (both fixed and random effect models are considered); these estimates for the NIM raise uncertainty for the non-inferiority conclusion in the PE trial.
Conclusion
The EINSTEIN trials used a NIM that was based on a meta-analysis of historical trials; the NIM used was the higher value estimated from the meta-analysis (when preserving 66% of the pooled treatment effect).8 Furthermore, the assessment of the meta-analysis calculation method suggested that the use of the random effects model would have been more appropriate. Estimating the NIMs based on the geometric scale raises some uncertainty about the non-inferiority conclusion of the EINSTEIN PE trial; the EINSTEIN DVT trial was not affected by the new NIMs.
