Study Design

One systematic review (with an embedded network meta-analysis),¹⁸ one RCT¹⁹ and one set of guidelines²⁰ were included in this review. The systematic review and guidelines were published in 2018, whereas the RCT was published in 2014. The systematic review and the RCT included partially overlapping groups of patients but reported on different sets of outcomes. The RCT compared two active therapy groups with one group of patients treated with placebo while the systematic review compared one of these active therapy groups with one group of patients treated with placebo.

The systematic review¹⁸ was conducted as part of a larger review on glatiramer acetate and β-interferons that included patients with relapsing-remitting MS and secondary progressive MS along with CIS.¹⁸ The authors limited their analysis to the subset of patients with CIS and to specific treatment regimens. The authors searched multiple databases in February 2016 and conducted a network meta-analysis of data from five double-blinded RCTs with open label study extensions; the search time frame was not reported. Information relevant to this report was recorded in the double-blind portion of the studies prior to open label crossover of patients into alternate intervention groups (primarily, from placebo to active treatment). The RCTs were: Rebif flexible dosing in early MS (REFLEX), CHAMPS, Pakdaman, BENEFIT, and PRECISE RCTs. The analysis of the treatment effect was conducted in two stages: first, a conventional pairwise meta-analysis was completed wherein active therapies were compared individually against placebo and where active therapies were compared as a group against placebo. At the next stage, following an assessment of heterogeneity, summary estimates for pairwise comparisons were calculated for each outcome of interest using a network meta-analysis.

The RCT¹⁹ reported on results from the REFLEX trial. The authors compared two dosages of one drug and placebo in patients with CIS. They calculated treatment effects as ratios of outcome measures between pairs of interventions and conducted pairwise comparisons based on an adjusted negative binomial model. The authors conducted subgroup analyses based on age, incidence of gadolinium-enhancing (Gd+) lesions at baseline, incidence of T2 non-enhancing lesions at baseline, focality of the presentation and steroid use at the time of the first demyelinating episode.

The guidelines were developed by a panel of 12 physician and nurse members of the American Academy of Neurology (AAN), two members from the Consortium of Multiple Sclerosis Centers, and three adults with an MS diagnosis who represented patients.²⁰ The guideline developers followed the AAN’s guideline development process.²⁰ The process started with searches of the MEDLINE, CENTRAL, and EMBASE databases from inception to November 2016. The developers then extracted evidence of the clinical effectiveness of DMTs from RCTs (when possible) while they extracted evidence of harms from RCTs, cohort studies, case series, and case reports. They considered whether the DMTs were superior to placebo in decreasing the risk of conversion from CIS to MS. A subset of the panel consulted other guideline panelists to ascertain the importance of certain outcome measures. Each recommendation in the guidelines was assigned a level of obligation (A, B, C, U or R) based on confidence in the evidence, soundness of inference assuming all premises are true, acceptance of axiomatic principles, and anticipated magnitude of benefit relative to harms. The level of obligation rating was determined in three stages: initial level, mandatory modification, and optional modification. The initial level of obligation was rated level A if the developers had high confidence in the evidence, if 100% of the developers were convinced of the soundness of inference assuming all premises were true, and if 100% of the developers accepted the axiomatic principles. The initial level of obligation was rated level B if the developers had moderate confidence in the evidence, if at least 80% and less than 100% of the developers were convinced of the soundness of inference assuming all premises were true and accepted the axiomatic principles, and it at least 80% of the developers believed that evidence cited from rerated conditions was strong. The initial level of obligation was rated level C if the developers had low confidence in the evidence, and if at least 50% up to less than 80% of the developers were convinced of the soundness of inference assuming all premises were true, accepted the axiomatic principles, and believed that the evidence cited from rerated conditions was strong. The initial level of obligation was rated level U or R if the developers had very low confidence in the evidence, and if less than 50% of the developers were convinced of the soundness of inference assuming all premises were true, accepted the axiomatic principles, and believed that the evidence cited from rerated conditions was strong.

The level of obligation was modified to level A, B, C, or U if there was a large benefit relative to harm, moderate benefit relative to harm, small benefit relative to harm, or it was too close to call, respectively. The level of benefit was large if there was large benefit and no harm; moderate, if there was large benefit and minimal harm or moderate benefit and no harm; and small, if there was large benefit and moderate harm, moderate benefit and minimal harm, or small benefit and no harm. The benefit was determined too close to call if benefit and harm were substantially similar.

The developers could optionally downgrade the level of obligation based on the importance of the outcome, expected variation in patient preferences, financial burden relative to expected benefits, and availability of the intervention.

AAN staff sent invitations asking key stakeholders, including all AAN section members, and pertinent external physician and patient organizations, including the Consortium of Multiple Sclerosis Centers, the Multiple Sclerosis Association of America, the Multiple Sclerosis Coalition, and the National Multiple Sclerosis Society to review drafts of the guidelines. The drafts were reviewed by at least three AAN committees, a network of neurologists, peer reviewers solicited through the Neurology journal, and representatives from related fields. The panel modified the draft or updated the systematic review, accordingly. The guideline was reviewed by the AAN’s Guideline Development, Dissemination, and Implementation Subcommittee before and after the public comment period. Individuals with perceived conflicts of interest were engaged as advisors to help validate key questions, assess the scope of the literature search, identify seminal articles, and participate in the recommendation development process. To mitigate their influence, individuals with conflicts of interest were not permitted to review or rate evidence.

Country of Origin

The systematic review¹⁸ was conducted by authors in the United Kingdom and the RCT¹⁹ was conducted by authors in Italy. The guidelines²⁰ were developed in the United States.

Patient Population

All the studies reported on the treatment of people with a single clinical episode diagnosed as having CIS. The systematic review¹⁸ reported on 1845 patients with a single clinical event and evidence of clinically silent lesions based on magnetic resonance imaging (MRI). In one study that was included in the systematic review, 171 patients were enrolled in each arm, but 146 patients completed the study in each arm. Some outcome measurements were taken in 146 patients while others were reported for 171 patients in each arm.¹⁸ The RCT¹⁹ enrolled 517 patients between the ages of 24 and 37 years who had an Expanded Disability Status Scale (EDSS) score between 0 and 5, a CIS episode within 60 days prior to study entry, and at least two clinically silent lesions visible on a T2-weighted brain MRI scan. The lesions had to be at least 3 mm in diameter and at least one of them had to be ovoid, located within white matter, or located below the tentorium cerebelli of the brain. No exclusion criteria were disclosed. The guidelines²⁰ focused on DMTs in people with MS (including CIS and relapsed MS). While the systematic review¹⁸ and guidelines²⁰ covered patients with a range of MS diagnoses, only information relevant to patients with CIS was considered for this report.

Interventions and Comparators

The interventions of interest in the included studies were glatiramer acetate, IFN β-1a, and IFN β-1b. The systematic review¹⁸ included one RCT in which patients had 20 mg of glatiramer acetate (Copaxone) subcutaneously once a day, one in which patients received 44 mcg of IFN β-1a (Rebif) subcutaneously three times weekly, two in which patients received 30 mcg of IFN β-1a (Avonex) intramuscularly (IM) once a week, and another in which patients received 250 mcg of IFN β-1b (Betaferon, Extavia) subcutaneously every other day. The comparator in these studies was placebo.

The RCT reported on three groups of patients who received 44 mcg IFN β-1a (Rebif) subcutaneously three times a week, 44 mcg IFN β-1a (Rebif) subcutaneously once a week, or placebo.

The guideline developers considered a broad range of therapies, including but not limited to glatiramer acetate, IFNs, and steroids.²⁰

Outcomes

The clinical effectiveness of glatiramer acetate and β-IFNs relative to placebo was assessed by the authors of the systematic review,¹⁸ RCT,¹⁹ and the guidelines²⁰ through the following outcomes and outcome measures:

Systematic Review and Network Meta-Analysis

The authors of the systematic review¹⁸ registered a protocol on a publicly-available database and included the population, interventions, and outcomes of interest in the statement of objectives, suggesting that elements of the review had been determined before the review started. Literature was acquired from multiple databases and study selection, data extraction, and quality assessment were conducted in duplicate. The sources of funding of the primary studies were disclosed enabling an accurate assessment of potential sources of bias. These characteristics presumably enhanced the credibility of the systematic review but not sufficiently to offset the serious limitations that were observed.

The systematic review was conducted as part of a larger review of patients with relapsing– remitting MS, secondary progressive MS, and CIS and in which glatiramer acetate and all forms of IFNs were compared against each other or placebo/best supportive care, and where clinical outcomes were reported such as relapse rates, progression to multiple sclerosis, or disability progression as measured by the EDSS.

The relevance of the patient population, interventions, and outcomes that were included in the network meta-analysis is driven by the interests of decision-makers. The results from the network meta-analysis are relevant to decision-makers who are interested in the relative clinical effectiveness (i.e., time to CDMS) of 44 mcg of IFN β-1a taken subcutaneously thrice weekly, 250 mcg of IFN β-1b taken subcutaneously every other day, 30 mcg of IFN β-1a injected intramuscularly weekly and 20 mg of glatiramer acetate taken subcutaneously daily in patients with CIS and clinically-silent lesions based on MRI examinations, prior to open-label extension periods. Decision-makers who are interested in the clinical effectiveness of other doses and forms of interferon or other therapies, patients with relapsing–remitting MS, secondary progressive MS, and other outcomes may find this network meta-analysis irrelevant. It is unclear whether the results of the network meta-analysis will hold in Canadian settings given that data was collected from patients being treated with multiple different regimens and in different countries.

Regarding credibility of the network meta-analysis, the authors attempted to identify all relevant RCTs by conducting searches of multiple databases; however they omitted some details of their literature searches such as keywords, results from a group of patients that were treated with 44 mcg of IFN β-1a weekly in the REFLEX trial, and outcomes such as change in lesion size following treatment.¹⁹ The authors provided insufficient information to ascertain whether statistical methods were used that preserved within-study randomization. No rationale was given for the use of a random-effects model. Regarding reporting quality and transparency, the authors reported individual study results and results of direct comparisons with measures of uncertainty (for some outcomes). The effect of important patient characteristics on treatment effects was not reported. The authors reported that they conducted an appraisal of the quality of the included RCTs, but they did not provide their findings nor results of an overall assessment of the quality of the body of evidence for each outcome. One of eight authors and a clinician received funding from the manufacturer of one of the therapies under evaluation, giving the manufacturer an opportunity to exert undue influence in exchange for financial compensation. The treatment effects of four different DMTs regimen were pooled in the meta-analysis. Since the DMTs might have unique mechanisms and effects, the pooled DMT comparison versus placebo is of debatable benefit.

Randomized controlled trial

The RCT¹⁹ exhibited some strengths that similarly were insufficient to offset its serious limitations. Registration of the RCT on a publicly-accessible database suggests that the authors were transparent in their research and minimized patient selection. The authors clearly described their objective and main outcomes. The patients and outcome assessors were blinded to the treatment options, accurate outcome measures were used, and patients were recruited from the same population over the same time frame. Estimates of random variability were reported in the form of 95% confidence intervals for the main outcomes and the statistical significance of the estimated treatment effects were indicated with P values. The following limitations were observed.

The authors omitted details when describing their population, study group sizes, interventions, potential confounders and main findings as follows: descriptions of patient characteristics were limited to age, sex, steroid use, disability score, and number of MRI-sensitive lesions. Importantly, it is unclear how many patients were enrolled at the time that measurements were recorded, suggestive of reporting bias. The authors indicate in their text that 146, 156, and 146 patients, respectively completed the study in the 44 mcg SC IFN β-1a thrice weekly, 30 mcg SC IFN β-1a weekly, and placebo groups. However, a figure in the article suggests that at 24 months, the number of patients in each group was 113, 119, and 88, respectively. The number of patients who converted to MS in each group prior to the follow-up timepoint was not disclosed. It is unclear how many patients in each group provided viable outcome measurements. Subgroup analyses were conducted for age, number of Gd+ lesions and number of T2 non-enhancing lesions; however, the choices of thresholds for these parameters were not justified suggesting post-hoc analyses may have been conducted. Regarding reporting of findings, the estimates of treatment effect were determined through pairwise comparisons based on an adjusted negative binomial model that was not described. As such, it is unclear how the point estimates were derived. The authors indicated that they assessed the quality and potential likelihood of bias of the included RCTs using the Cochrane Risk of Bias tool but did not discuss the quality of the body of evidence for each clinical outcome.

External validity could not be evaluated given that details of the recruitment process were unavailable. It does appear that patients were selected to be at high risk of conversion to MS. The apparent use of stringent inclusion criteria means the generalizability of the results may be limited because the patients may not be representative of the general population of patients with CIS. Other than conversion to MS, the rates and reasons for dropping out in less than 24 months were not discussed; therefore, it is unclear whether the patients who were prepared to participate were representative of the population of patients from which they were recruited. Details of the staff and location of treatment were missing, precluding an assessment.

Some elements that are required to assess the risk to internal validity were missing such as the number of patients who converted to MS prior to 24 months and information on whether any adjustment was made to account for different lengths of follow-up. Finally, the authors did not discuss compliance with the intervention or statistical power of the study. Some of the missing information may be available in companion reports on the REFLEX trial.

Guideline

The included set of guidelines had more strengths than limitations.²⁰ The scope and purpose were well-defined, stakeholders appeared to be inclusive although it was not possible to assess whether all relevant professional groups were represented. All elements of a rigorous development process were present such as systematic selection of evidence, clear descriptions of the strengths and limitations of the included RCTs and the process through which recommendations were formulated, explicit links between the recommendations and the supporting evidence, an outline of an external peer-review process, and guidance for future updates. The key recommendations were specific, unambiguous and clearly identifiable.

The guideline’s quality was weakest in the applicability domain. The developers did not provide advice on and/or tools for implementing their recommendations, nor did they describe facilitators and barriers to implementation. They also did not discuss potential resource implications of applying the recommendations nor monitoring and/or auditing criteria. Given the evidence on the use of glatiramer acetate and β-INF in treating patients with CIS was limited to four RCTs, users of the guidelines will likely need substantial support in implementing the recommendations.

Clinical effectiveness at 24 months and/or 36 months

One systematic review¹⁸ and one RCT¹⁹ provided evidence on the clinical effectiveness of glatiramer acetate, IFN β-1a, and IFN β-1b in people with CIS. The RCT reported on a set of patients who were included in the systematic review, i.e., patients who were treated with IFN β-1a once a week.¹⁹ The guideline developers considered evidence from four of the five RCTs that were included in the systematic review.²⁰

Safety

The systematic review reported mixed results on the incidence of discontinuation due to adverse events.¹⁸ Comparisons of the proportion of patients who discontinued the study due to adverse events favoured 44 mcg of SC IFN β-1a thrice weekly and 30 mcg of IM IFN β-1a weekly over placebo. Similar calculations favoured placebo over 250 mcg of SC IFN β-1b and 20 mg of SC glatiramer acetate daily. The authors did not quantitatively synthesize the evidence from the RCTs nor did they asses the statistical significance of the differences in outcome measures between the active therapies and placebo. Further, the authors did not describe the specific events that led to patients discontinuing treatment. The authors of the RCT did not identify any safety signals, in direct contradiction to the findings on the incidence of discontinuation due to adverse events in the systematic review.¹⁹

There was no evidence identified on incidence of progression to CDMS, mortality, hospitalizations, quality of life, disability, time to disability, long-term disability progression, or relapse.

Guidelines

The evidence on clinical effectiveness for the single set of guidelines was extracted from four of the five RCTs that were included in the systematic review.¹⁸ Appendix 5 outlines the overlap among the studies that are included in this review.

The evidence favoured glatiramer acetate, IFN β-1a, and IFN β-1b over placebo. Relative to placebo, the risk of conversion to MS over 3 years was significantly lower with 20 mg of glatiramer acetate taken subcutaneously daily, with an RR of 0.58 (95% CI, 0.44 0.75). Relative to placebo, the risk of conversion to MS over 2 years was significantly lower with 44 mcg of IFN β-1a taken subcutaneously thrice weekly, with an RR of 0.55 (95% CI, 0.38 0.78). Relative to placebo, the risk of conversion to MS over 3 years was significantly lower with 30 mcg IFN β-1a injected intramuscularly weekly, with an RR of 0.71 (95% CI, 0.56 0.89%), Relative to placebo, the risk of conversion to MS over 2 years was significantly lower with 250 mcg of IFN β-1b taken subcutaneously every other day, with an RR of 0.59 (95% CI, 0.46 0.76). The risk of harms from initiating DMTs (such as adverse events, major adverse events, and burden of taking a long-term medication) relative to the benefit of reducing relapse rate in patients with CIS or relapsing forms of MS who have not had relapses in two or more years and do not have active new MRI lesion activity on recent imaging is unknown.

The guidelines included four relevant recommendations:

• “Clinicians should discuss the benefits and risks of DMTs for people with a single clinical demyelinating event with [two] or more brain lesions that have imaging characteristics consistent with MS” (page 174)²⁰
• “After discussing the risks and benefits, clinicians should prescribe DMT to people with a single clinical demyelinating event and two or more brain lesions characteristic of MS who decide they want [DMT]” (page 175)²⁰
• “Clinicians may recommend serial imaging at least annually for the first five years and close follow-up rather than initiating DMT in people with CIS or relapsing forms of MS who are not on DMT, have not had relapses in the preceding two years, and do not have active new MRI lesion activity on recent imaging” (p 176)²⁰
• “Clinicians should [compare] the associated risks of continuing DMTs [with] those risks that are associated with stopping DMTs in people with CIS (who have not been diagnosed with MS).” (page 212)²⁰ There was some uncertainty in the level of consensus achieved for this recommendation.

Recommendations on switching between therapies specifically for patients with CIS were not found although there were recommendations for patients diagnosed with MS.