Background:

Patent foramen ovale (PFO)-associated strokes comprise approximately 10% of ischemic strokes in adults aged 18 to 60 years. Despite the overall beneficial effects of closure device placement in patients with a first PFO-associated cerebral ischemic event, the best treatment option for any individual patient encountered in routine clinical practice is often quite unclear.

Objective:

The objective of this study was to evaluate the heterogeneity of treatment effect of PFO closure on stroke recurrence based on previously developed scoring systems.

Methods:

Individual patient data were pooled from 6 randomized clinical trials that compared PFO closure plus medical therapy vs medical therapy alone in patients with PFO-associated stroke, which involved a total of 3740 participants. The trials were conducted worldwide from 2000 to 2017. Comparisons were made between PFO closure plus medical therapy vs medical therapy alone. Subgroup analyses used the Risk of Paradoxical Embolism (RoPE) score (a 10-point score in which higher scores reflect younger age and the absence of vascular risk factors) and the PFO-Associated Stroke Causal Likelihood (PASCAL) algorithm, which combines the RoPE score with high-risk PFO features (either an atrial septal aneurysm or a large shunt) to classify patients into 3 categories of causal relatedness: “unlikely,” “possible,” and “probable.” The main outcome was ischemic stroke.

Results:

Over a median follow-up of 57 months (interquartile range, 24-64 months), 121 outcomes occurred in 3740 patients. The annualized incidence of stroke with medical therapy was 1.09% (95% CI, 0.88%-1.36%) and with device closure was 0.47% (95% CI, 0.35%-0.65%); the adjusted hazard ratio (HR) was 0.41 (95% CI, 0.27-0.60). Subgroup analyses showed statistically significant interaction effects. Patients with low vs high RoPE score had HRs of 0.61 (95% CI, 0.37-1.00) and 0.21 (95% CI, 0.11-0.42), respectively (P for interaction = .02). Patients classified under PASCAL as unlikely, possible, and probable had HRs of 1.14 (95% CI, 0.53-2.46), 0.38 (95% CI, 0.22-0.65), and 0.10 (95% CI, 0.03-0.35), respectively (P for interaction = .003). The 2-year absolute risk reduction was −0.7% (95% CI, −4.0% to 2.6%), 2.1% (95% CI, 0.6%-3.6%), and 2.1% (95% CI, 0.9%-3.4%) in the unlikely, possible, and probable PASCAL categories, respectively. Device-associated adverse events were generally higher among patients classified as unlikely; the absolute risk increases in atrial fibrillation beyond day 45 postrandomization with device were 4.41% (95% CI, 1.02%-7.80%), 1.53% (95% CI, 0.33%-2.72%), 0.65% (95% CI, −0.41% to 1.71%) in the unlikely, possible, and probable PASCAL categories, respectively.

Conclusions:

Among patients aged 18 to 60 years with PFO-associated stroke, risk reduction for recurrent stroke with device closure varied across groups classified by their probabilities that the stroke was causally related to the PFO. Application of these classification systems has the potential to guide individualized decisions regarding the selection of device closure vs medical therapy, supporting patient-centered decision-making for patients with PFO-associated cerebral ischemic events.

Limitations:

Some limitations of the study were the following: data were missing with respect to functional outcomes with recurrent stroke; trials had heterogenous definitions of key variables; the original PASCAL classification could not be evaluated; and several questions remain unaddressed, such as the best type of antithrombotic therapy, the role of new PFO devices, and the role of closure for patients older than 60 years.

Clinical Uncertainty and Questions to Be Addressed

Despite the overall beneficial effects of closure device placement in patients with a first PFO-associated cerebral ischemic event, the best treatment option for any individual patient encountered in routine clinical practice is often quite unclear. When a patient has a stroke with an unclear cause (called a cryptogenic stroke), an echocardiogram may be performed to see if the patient has a PFO. Patent foramen ovale is present in ~25% of the general population, and patients with a PFO can either have a stroke through a PFO-related mechanism (eg, a paradoxical embolism) or through any other occult mechanism (eg, paroxysmal AF or minimally stenosing cervicocerebral atherosclerotic plaque). Patent foramen ovale closure is highly unlikely to reduce recurrence risk in patients whose index event has a cause unrelated to PFO, but it is typically impossible to know with certainty the cause in any individual with a PFO and cryptogenic stroke. Studies to date have not been able to address this individual patient–level clinical uncertainty.

Study-level analysis of RCTs generally only bring forward for guidance the broad reference class of all patients qualifying for a trial. In study-level data subgroup analyses, several different patient characteristics modified or tended to modify the magnitude and even the presence of benefit of 1 therapy over another. Conventional (1-variable-at-a-time) subgroup analysis as reported in trials is frequently used to explore heterogeneity of treatment effects (HTE), but these have well-known issues both with regard to credibility and applicability to individuals, because patients differ in many ways simultaneously.^19-21 In the case of PFO-associated stroke, individuals may differ from one another in the probability that a PFO was causally related to the index event²² and in the likelihood of a recurrent event. The coarse information so far reported from the 6 previous RCTs as well as from study-level meta-analyses of these RCTs is insufficient to fully inform the best treatment choice for each individual patient.^19-21

To address these limitations and to optimize individual decision-making for patients with PFO-associated stroke, the trialists who conducted all 6 completed RCTs of secondary prevention therapies for PFO-associated stroke have joined in this study to pool the data sets from all studies and to perform an individual patient data (IPD) meta-analysis.

Scoring Systems That Predict the Causal Relationship of PFO to Stroke

To evaluate the HTE of PFO closure on stroke recurrence, we use previously developed scoring systems that help estimate the probability that a PFO discovered in the setting of a cryptogenic stroke is likely to be causally related to the stroke. The first system is the Risk of Paradoxical Embolism (RoPE) score,²² and the second is the PFO-Associated Stroke Causal Likelihood (PASCAL) classification system.⁵

The RoPE Score

The RoPE scoring system is depicted in Table 1 and explicated in Appendix A2. Briefly, the RoPE score provides an estimate of the probability that a PFO discovered in the setting of an otherwise-cryptogenic ischemic stroke is the cause of the stroke rather than an incidental finding, with a higher RoPE score corresponding to a higher probability. The RoPE score is based on 2 insights: (1) the prevalence of a PFO among patients with cryptogenic stroke (compared with its prevalence in the general population) can be used, via Bayes' theorem, to estimate an average attributable fraction (ie, the proportion of PFOs that are pathogenic rather than incidental), and (2) the presence or absence of a PFO in a patient with a cryptogenic stroke is predictable based on patient characteristics—so that a “patient-specific” attributable fraction can be estimated based on the probability of discovering a PFO conditional on patient characteristics. Intuitively, a PFO-related stroke is more likely in younger patients, in the absence of vascular risk factors, and in the presence of a superficial infarct on neuroimaging. In theory, because closure would reliably prevent strokes caused by paradoxical embolism and only strokes caused by PFO, the attributable fraction would be assumed to correspond to the relative risk reduction (RRR) of closure in preventing a future stroke.²³

Table 1

The RoPE Score and PASCAL Classifications.

The ROPE score was developed for use on 9 different databases²²; although some of these data were from Europe and thus not fully representative of ethnically and racially diverse American population, we note that the score performed consistently across all 9 databases, including those with more diverse samples (eg, Northern Manhattan Stroke Study [NOMASS]).²⁴

The RoPE score has been externally validated to predict the presence of a PFO in the cryptogenic stroke population.^25,26 However, the RoPE score has 2 important limitations: (1) the methods to derive the RoPE score did not permit the inclusion of high-risk features of the PFO, and (2) patients with higher RoPE scores have lower stroke recurrence rates. Thus, the RoPE score may not provide comprehensive information for patient selection.^5,8

Overview of Study Goals and Specific Aims

This research encompasses an updated, pooled, IPD meta-analysis (IPDMA) by a collaboration of the trialists of all 6 completed RCTs comparing PFO closure with percutaneous devices plus medical therapy vs medical therapy alone,^31-33 and examines factors that influence which therapy is best for whom. We test the influence of a wide range of patient demographic, clinical, and cardiac (eg, size of right-to-left shunt, presence of ASA) features on the effects of device closure vs medical therapy. In addition, we assess the incremental added value for effect modification modeling of 2 existing multivariable scales/algorithms to grade the causal relationship between a PFO and an index cerebral ischemic event: the RoPE score and the PASCAL classification grade. This IPD pooled analysis was undertaken, motivated by new methods proposed for predictive HTE analyses, combining many covariates, to narrow the reference class for each individual to more granular, deeply similar patients.^19,20 Individual patient data meta-analysis has several advantages over study-level meta-analysis,³⁴ including standardization of analyses across studies, better handling of missing data via appropriate statistical methods, the ability to estimate conditional treatment effects (often with greater statistical power for tests of the null hypothesis than with unadjusted analyses of time-to-event outcomes^6,12,14), and the opportunity to assess HTE across subgroups of interest—the main goal of this study. In particular, IPDMA is an ideal substrate for risk modeling approaches to HTE analysis because it is better powered than individual trials and there is typically a greater degree of patient-level risk heterogeneity than in any given trial^35,36:

• Specific aim 1. Perform an IPDMA comparing the broad strategy of placement of a PFO closure device plus best medical therapy with the broad strategy of best medical therapy alone for the prevention of recurrent stroke, including an exploration of clinically relevant subgroups.
• Specific aim 2. Examine whether (a) the RoPE score modifies the relative effect of device closure vs medical therapy for the prevention of recurrent stroke and (b) whether the RoPE-estimated attributable fraction correlates with the RRR associated with closure across levels of the RoPE score.
• Specific aim 3. Assess whether the PASCAL classification grade (which integrates the RoPE score with high-risk PFO features) modifies the relative effects of device closure and compare the net clinical benefit (ie, value of a model to improve decision-making) of applying the RoPE score and the PASCAL classification grade for treatment selection against a default “treat-all” strategy.

Study Overview

We performed a pooled IPDMA through a collaboration of the trialists of all 6 completed RCTs comparing PFO closure with percutaneous devices plus medical therapy vs medical therapy alone^31-33 and examined factors that influence which therapy is best for whom. We tested the influence of a wide range of patient demographic, clinical, and cardiac features on the effects of device closure vs medical therapy. In addition, we assessed the incremental added value for effect modification modeling of 2 existing multivariable scales/algorithms to grade the causal relationship between a PFO and an index cerebral ischemic event: the RoPE score and the PASCAL classification grade.

Study Setting/Participants

The study investigators established the SCOPE Consortium to undertake meta-analysis of pooled IPD. Study methods adhered to the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) IPD guidelines, and the protocol was registered on the international prospective register of systematic reviews, PROSPERO (CRD42020186537).³⁷ The collaboration included all randomized phase 3 trials comparing PFO closure with medical therapy for recurrent stroke prevention published by September 2021. Trial methodology was assessed using the Risk of Bias 2.0 tool. Investigators were contacted, and data were collected and harmonized (for details, see Appendix A1 and Kent et al³⁸). The data were harmonized and analyzed by 2 statisticians at the Predictive Analytics and Comparative Effectiveness (PACE) Center and Tufts Medical Center to ensure they accurately matched the values reported by the trials. Appendix Table 1 in Appendix A1 lists the variables that were harmonized across trials. Though many variables were defined consistently between trials, Appendix A5 describes variables that required harmonizing, including ASA and shunt size. When disagreement arose or database definitions were inconsistent, 2 clinicians adjudicated the variables. Some key variables were not present in certain databases; thus, harmonization also required rereading of REDUCE echocardiograms and MRIs, which was performed by our team blinded to treatment assignment and outcome. The PC Trial was missing a key RoPE variable (superficial infarct on neuroimaging), and neuroimages from this trial could not be obtained. The following 6 trials are included in this IPDMA: the CLOSE trial,¹⁰ The CLOSURE I trial,¹¹ DEFENSE-PFO trial,¹² the PC Trial¹³ the REDUCE trial,¹⁴ and the RESPECT trial^15,16 (Appendix Table 5). All trials compared the broad strategy of PFO closure device plus best medical therapy with the broad strategy of best medical therapy alone to prevent stroke recurrence in patients with PFO-associated cerebral ischemia. Some heterogeneity in the risk of bias is anticipated because of trial design differences and data missingness.^39-41 Typical of device or operative trials, all studies had Prospective Randomized Open Blinded End-point (PROBE) designs, with the exception of the DEFENSE-PFO study, which did not have blinded adjudication of outcome. Th details of these trials are described briefly in Appendix B2 and in the sections below.

Based on a systematic search performed of Medline and Embase through September 2021, these studies represent the totality of available randomized evidence on the use of percutaneous implanted devices for PFO closure vs medical therapy in patients with PFO-associated cerebral ischemic events (Appendix A1). All contributing RCTs were asked to provide the individual patient–level study data shown in Appendix Table 1. Data entered into the central SCOPE database were a limited data set, with all high-level patient identifiers removed. All data were collected under the aegis and supervision of the SCOPE steering committee, and were integrated and stored at the PACE Center at Tufts Medical Center in Boston, Massachusetts.

Interventions and Comparators or Controls

All trials compared PFO closure with standard medical therapy. Treatment assignment was randomized by computer-generated, pseudorandom numbers. Patent foramen ovale closure was attempted with various devices, specifically detailed below:

• CLOSE used a wide variety of devices including, but not limited to, Amplatzer PFO Occluder (AGA Medical), Intrasept PFO occluder (Cardia), Premere Patent Foramen Ovale Closure System (St Jude Medical), STARFlex septal occlusion system (NMT Medical Inc), Amplatzer Cribriform Occluder (AGA Medical), and Figulla Flex II PFO Occluder (Occlutech, Inc). All patients who underwent PFO closure received dual-antiplatelet therapy (75 mg of aspirin plus 75 mg of clopidogrel daily) for 3 months, followed by single-antiplatelet therapy.
• CLOSURE I used the STARFlex septal closure system. After closure, all patients were given a standard antiplatelet regimen of clopidogrel 75 mg daily for 6 months and aspirin 81 or 325 mg daily for 2 years.
• DEFENSE-PFO used a single device (Amplatzer PFO Occluder, St Jude) for closure. Patients who underwent PFO closure were generally recommended to start a dual-antiplatelet regimen (aspirin 100 mg daily plus clopidogrel 75 mg daily) for at least 6 months after the procedure. However, the local investigator or attending neurologists could choose to continue either antiplatelet therapy or anticoagulation based on the individual's risk/benefit ratio.
• The PC Trial used the Amplatzer PFO Occluder For patients randomized to closure, antithrombotic treatment was recommended, including acetylsalicylic acid at a dose of 100 to 325 mg per day for at least 5 to 6 months, as well as ticlopidine at a dose of 250 to 500 mg per day or clopidogrel at a dose of 75 to 150 mg per day for 1 to 6 months. For patients with intolerance to acetylsalicylic acid, ticlopidine or clopidogrel alone was recommended.
• REDUCE used either the Helex septal occluder (HELEX; W.L. Gore and Associates) device (implanted through late 2012) or the Cardioform Septal Occluder (GSO; W.L. Gore and Associates). All patients received antiplatelet therapy chosen by the local investigator. Patients in the PFO closure group were treated with 1 dose of clopidogrel (300 mg) before or immediately after the procedure, if they were not already receiving clopidogrel, followed by 75 mg daily for 3 days, and then resumed or started the chosen antiplatelet therapy option.
• RESPECT used the Amplatzer PFO Occluder. Patients assigned to closure also received 81 to 325 mg of aspirin and clopidogrel for 1 month after placement of the device, followed by aspirin monotherapy for 5 months. Subsequently, antiplatelet therapy was administered at the discretion of the site investigator.

The control groups received the following treatments:

• CLOSE used oral anticoagulants with vitamin K antagonists (with a target international normalized ratio [INR] of 2-3) or with direct oral anticoagulants (for patients assigned to oral anticoagulants). Patients assigned to antiplatelet therapy alone could receive aspirin, clopidogrel, or aspirin combined with extended-release dipyridamole.
• CLOSURE I used 1 of the following: warfarin with a target INR of 2 to 3 with an ideal target of 2.5; aspirin 325 mg daily; aspirin 81 mg daily only, allowed for documented gastrointestinal intolerance; or aspirin 81 mg daily with warfarin. Clopidogrel, ticlopidine, and aspirin plus extended-release dipyridamole were not allowed in the medical arm.
• For DEFENSE-PFO, all patients received either antiplatelet therapy or anticoagulation chosen by the local investigator. Antiplatelet therapy included aspirin, aspirin plus clopidogrel (75 mg per day), or aspirin plus cilostazol (200 mg per day). Warfarin was used to maintain the target INR of 2 to 3.
• For the PC Trial, antithrombotic treatment in the medical group was left at the discretion of the treating physician, which could include antiplatelet or oral anticoagulation.
• For REDUCE, all patients received antiplatelet therapy chosen by the local investigator. Antiplatelet therapy could consist of aspirin alone (75-325 mg once daily), a combination of aspirin (50-100 mg daily) and dipyridamole (225-400 mg daily), or clopidogrel (75 mg once daily). Other combinations of antiplatelet drugs and anticoagulants were not permitted. The chosen antiplatelet therapy could be initiated immediately after randomization.
• RESPECT used 1 of the following: aspirin, warfarin, clopidogrel, or aspirin combined with extended-release dipyridamole. Aspirin with clopidogrel was also originally permitted but was eliminated in 2006 to conform to a change in secondary stroke prevention guidelines.⁴²

The populations and treatment comparisons evaluated across these 6 trials were judged to be similar enough to justify making quantitative pooled analyses of their data sets.

Study Outcomes

The primary efficacy end point was recurrent ischemic stroke: an acute neurological deficit, presumed to be caused by focal ischemia, and either symptoms persisting for at least 24 hours or symptoms persisting less than 24 hours but associated with neuroimaging findings of a new, neuroanatomically relevant infarct.^43,44 This definition is the preferred definition for clinical care and for research trial stroke end points, as per national recommendations from the American Heart/Stroke Association, the National Institutes of Health,⁴³ and the US Food and Drug Administration.⁴⁴

The secondary efficacy outcomes were (1) recurrent PFO-associated ischemic stroke (recurrent ischemic strokes adjudicated as not being attributable to another mechanism by investigators of the individual trials); (2) the composite of recurrent ischemic stroke or early (periprocedural or equivalent medical therapy time frame) all-cause mortality; (3) the composite of recurrent ischemic stroke, early all-cause mortality, or any vascular death; (4) the composite of recurrent ischemic stroke, TIA, or vascular death; (5) disability-worsening recurrent ischemic stroke; and (6) any recurrent stroke, ischemic or hemorrhagic.

Disability-worsening stroke was defined as a new stroke associated with any increase in the modified Rankin score at day 30 or longer poststroke. For subgroup analysis, in addition to the primary end point, only the secondary end point with the greatest number of events was examined: the composite of recurrent ischemic stroke, TIA, or vascular death.

For technical efficacy outcomes, we report effective PFO closure at 6 to 18 months (latest observation time point), defined as no or only trace residual shunting as the lead technical efficacy outcome. We also report complete PFO closure at 6 to 18 months (latest observation time point), defined as no residual shunting.

Five safety outcomes were examined, as defined in the primary trials: all serious adverse events (SAEs); major vascular procedural complication; AF; major bleeding episode; and venous thromboembolism (VTE; deep venous thrombosis/pulmonary embolism). Atrial fibrillation was analyzed in 2 ways: (1) any AF and (2) AF present any time beyond the first 45 days postrandomization (to exclude transient periprocedural events). The following components of procedure-related adverse events are also reported: access site hemorrhage, retroperitoneal hemorrhage, cardiac tamponade, and cardiac perforation.

Sample-Size Calculations and Power

No sample-size calculations were performed for the primary analysis comparing a closure device with medical therapy because we had no control over the design of the studies included in our analyses, the studies represent the totality of available randomized evidence on PFO closure, and each study was planned based on separate prospective power calculations.

Analytical and Statistical Approaches

All analyses were performed on the pooled data. The primary analyses for efficacy outcomes assessed patients according to the treatment group to which they were randomized (ie, intention to treat [ITT]). Three additional analyses were conducted in (1) patients according to the treatment they actually received if any crossover occurred (ie, “as-treated,” defined in Appendix A4), (2) patients without major protocol deviations (ie, “per-protocol,” defined in Appendix A4), and (3) analysis of the ITT population, using instrumental variable analysis to further control for potential pre- and postrandomization sources of bias in estimating treatment effects (ie, “contamination-adjusted ITT analysis”).⁴⁵ These analyses were adjusted for prerandomization covariates.⁴⁶ Safety outcomes were analyzed in the as-treated population only.

The secondary efficacy outcomes were analyzed using sequential gatekeeping⁴⁷ in the following order: (1) recurrent PFO-associated ischemic stroke; (2) the composite of recurrent ischemic stroke or early all-cause mortality; (3) the composite of recurrent ischemic stroke, early all-cause mortality, or any vascular death; (4) the composite of recurrent ischemic stroke, TIA, or vascular death; (5) disability-worsening recurrent ischemic stroke; and (6) any recurrent stroke, ischemic or hemorrhagic. The sequential gatekeeping strategy tests a prespecified hierarchy of outcomes and stops when an outcome test is not statistically significant, to control for multiplicity. Secondary efficacy outcomes that were unavailable uniformly across all trials were to be removed from the gatekeeper hierarchy and examined in exploratory analyses where available.

For all time-to-event outcomes, the equality of the survivor functions was assessed using a stratified (by trial) log-rank test.^22,25 Kaplan-Meier estimates were obtained at 6, 12, 24, and 60 months for each treatment group. After confirming no violation of proportional hazards assumptions (see Appendix A1), effects were estimated using Cox proportional hazards regression with a study-specific random effect.⁴⁸ In this 1-stage analysis, a study-specific random effect was used to account for within-study homogeneity in outcomes and a fixed treatment effect.⁴⁹ The CLOSE trial, in which some patients were randomized to antiplatelet vs device and others to anticoagulant vs device, was treated as 2 trials. The Breslow method was used for tied survival times.⁴⁸ Safety analyses were based on comparisons of event proportions between treatment groups, using the Cochran-Mantel-Haenszel test (stratified by trial).^50,51 For all hypothesis-testing analyses, a 2-sided significance threshold of P = .05 was used, without adjustments for multiple testing.⁵² All analyses were conducted using SAS (version 9.4) and R (version 4.0.2).

The primary efficacy analysis was adjusted for the following covariates: age, sex, prior myocardial infarction, diabetes, hypertension, hyperlipidemia, prior stroke or TIA, smoking status, index event (stroke vs TIA), ASA (≥10 mm of excursion from midline; definition in Appendix A5), PFO shunt size (large vs small; definition in Appendix A5), and presence vs absence of a visible superficial infarction on neuroimaging. This analysis yields a conditional average treatment effect, which is felt to be more appropriate for individual patient decisions.⁵³ As a stability analysis, the unadjusted effect estimate is also reported. Additional details of the statistical analysis are provided in Appendix A1.

Changes to the Original Study Protocol

The following protocol changes were made based on discussion at the February investigator meeting:

1. Classified safety outcome for AF as periprocedural vs persistent (based on presence or absence more than 45 days from randomization)
2. Made the primary analysis covariate adjusted for the prespecified variables (dependent on their uniform availability) of age, sex, coronary artery disease, diabetes, hypertension, hyperlipidemia, prior stroke, smoking status, index event (stroke vs TIA), hypermobile septum, and PFO shunt size (large vs small). This approach increases power, and the conditional average treatment effect is generally considered the estimate of interest for individual patient decision-making
3. Changed classification of large vs small shunt using a bubble cutoff of 20 instead of 10. This was largely based on availability across study databases
4. Added a dichotomous HTE analysis of the RoPE score, ≥7 vs < 7
5. Added an analysis of the correlation between the RoPE-estimated attributable fraction and the RRR. These items were added based on a recently completed analysis

The following revisions were accepted and incorporated into the protocol after receiving suggestions from investigators in the February meeting:

1. Extensive revisions of the initial background were made to be consistent with new nomenclature and paradigms reflecting a consensus group statement I (recently published in JAMA Neurology⁵) from the old “cryptogenic stroke with PFO” to the current “PFO-associated stroke” paradigm. The entire introduction was revised
2. Added the PASCAL grading system (also from our joint JAMA Neurology PFO-Associated Stroke Paper),⁵ which was assessed as a risk stratification tool in the IPDMA. This tool integrates the RoPE score with high-risk anatomic features of the PFO. As a third specific aim, we assessed whether the PASCAL classification grade modifies the relative effects of device closure and compared the net clinical benefit of applying the RoPE score and the PASCAL classification grade for treatment selection against a default treat-all strategy. We added an extensive section to the analysis plan to address this aim
3. Added technical efficacy outcomes (effective PFO closure, complete PFO closure)
4. Added analysis of outcomes of interest at 60 months, in addition to 6, 12, and 24 months
5. Revised analysis for per-protocol, as-treated (ie, got the device vs did not get the device), and contaminated-adjusted ITT analyses
6. Suggested revisions to the governance structure based on template from the Highly Effective Reperfusion evaluated in Multiple Endovascular Stroke Trials (HERMES) endovascular thrombectomy Trialist collaborative
7. Several minor corrections to the descriptions of the 6 RCTs

The following changes were made based on discussion at the trialist webinar:

1. Additional safety outcomes to be included (in addition to major vascular procedural complications, AF, and major bleeding episodes) and defined according to the definitions of the primary studies: access site hemorrhage, retroperitoneal hemorrhage, pericardial tamponade, cardiac perforation, VTE, and all SAEs
2. It was suggested that, rather than having just a single secondary clinical efficacy end point, we consider the following secondary clinical efficacy end points (after the primary efficacy end point of recurrent ischemic stroke): recurrent PFO-associated ischemic stroke; recurrent ischemic stroke or early all-cause mortality; recurrent ischemic stroke, early all-cause mortality, or any vascular death; recurrent ischemic stroke, TIA, or vascular death; disabling recurrent ischemic stroke; and any recurrent stroke, ischemic or hemorrhagic. These secondary clinical efficacy end points would be tested using the serial “gatekeeping” strategy for multiplicity control, allowing testing of multiple secondary end points with rigor and statistical power. Secondary effectiveness outcomes that were unavailable uniformly across all trials were removed from the gatekeeper analyses and examined in exploratory analyses with the trials in which they are available. We note that disabling ischemic stroke and PFO-associated stroke may not be ascertainable across all data sets and may have to be removed as secondary clinical end points
3. Different approaches were suggested for sensitivity analyses to handle missing outcome data. In the primary analysis, we assumed that patients who exited the trials early have outcome events that are noninformative under the MAR assumption and censored these patients at last follow-up. This assumes that censoring occurs independent from the possibly unobserved time to event: that is, the possibly unknown true time to the event for a patient is the same regardless of whether it is actually observed (or whether censoring occurs before it). Nevertheless, this MAR assumption may not hold, and treatment effect estimates may be sensitive to this especially when censoring is imbalanced with respect to treatment assignment. Thus, if rates of patients exiting the trials early are substantial (ie, >10% of all the expected follow-up time), a sensitivity analysis was performed

Disabling stroke was defined as an increase in the modified Rankin Scale (mRS) score of at least 1 point from the last measured mRS score before a stroke recurrence. The worsened mRS score must have been ascertained at least 30 days from the stroke recurrence. If disabling stroke could not be ascertained across databases, it was removed as a secondary end point in the main analysis; an exploratory analysis was considered.

Primary and Secondary Efficacy Outcomes

During a median follow-up time of 57 months, a total of 121 primary end point ischemic stroke events occurred in the pooled study population. Treatment with PFO closure was associated with reduced incidence of recurrent ischemic stroke (Table 3, Figure 1). The annualized incidence of stroke with medical therapy was 1.09% (95% CI, 0.88%-1.36%), and that with device closure was 0.47% (95% CI, 0.35%-0.65%) (adjusted HR, 0.41 [95% CI, 0.27-0.60]; P < .001). Secondary outcomes showed results that were consistent with the primary outcome (Table 3), except for disability-worsening stroke, which was limited because of missingness.

Table 3

Primary and Secondary Efficacy Outcomes.

Figure 1

Recurrent Ischemic Stroke Kaplan-Meier Curve.

Stability of the Main Results

The main results for the primary outcome were robust to alternative analytic approaches. Hazard ratios were similar for the unadjusted (Table 3), per-protocol (HR, 0.37 [95% CI, 0.24-0.57]), and as-treated (0.40 [95% CI, 0.27-0.59]) analyses. Leave-one-out analyses showed that no single trial was overly influential (Appendix B5); the adjusted HR for closure ranged from 0.32 (95% CI, 0.20-0.51) without CLOSURE to 0.45 (95% CI, 0.30-0.66) without CLOSE-A (CLOSE trial randomization group 2 with contraindications to oral anticoagulants, Appendix B4). Early exiting and retained patients were largely similar (Appendix B6), although patients who had early exit from the trials posttreatment more often had prior stroke, a superficial infarct, and a large shunt. In the multiple-imputation analysis with covariate adjustment, the HR for ischemic stroke with PFO closure vs medical therapy was 0.41 (95% CI, 0.26-0.64; P < .001). The tipping-point analysis showed robustness to missing data: imputed outcomes for patients not followed to the end of each trial would have to be approximately 2-fold higher in the device than in the medical group to nullify the significance of the main effect (Appendix B7). A 1-stage meta-analysis of the 6 studies using the inverse variance method and restricted maximum likelihood estimator revealed the following values: HR for fixed effects model of 0.54 (95% CI, 0.36-0.81), HR for random effects model of 0.52 (95% CI, 0.30-0.88), and test of heterogeneity P = .44.

Safety Outcomes

Safety analyses are shown in Table 4. Atrial fibrillation was significantly higher in the closure group (adjusted relative risk [RR], 4.54 [95% CI, 2.78-7.39]), but 46% (50/109) of the events were transient, occurring only in the first 45 days postrandomization. Beyond this periprocedural period, the rate of AF over a median follow-up of 57 months was 5.0% with device and 1.1% with medical therapy (adjusted RR, 2.60 [95% CI, 1.44-4.70]).

Table 4

Safety Outcomes.

RoPE Score, PASCAL Grade, and HTE

As described above, we hypothesized that the RoPE score would stratify the population by their attributable fraction and their likelihood of benefit from PFO closure. The RoPE score × treatment interaction was significant (P < .01), using the full 10-point RoPE score with the neuroimaging variable imputed for the PC Trial. Results stratified by RoPE scores of ≥7 vs <7 are shown in Figure 2, indicating a strong interaction (P = .02) on the HR scale (Figure 2A); patients with low vs high RoPE scores had HRs of 0.61 (95% CI, 0.37-1.00) and 0.21 (95% CI, 0.11-0.42), respectively. The rates for risk of stroke in the first 2 years for patients with a low RoPE score were 4.0% (95% CI, 2.5%-5.5%) and 2.9% (95% CI, 1.6-4.2) (absolute risk reduction [ARR], 1.1% [95% CI, −0.9% to 3.1%]) for the medical therapy and device groups, respectively, and 2.6% (95% CI, 1.7%-3.6%) and 0.6% (95% CI, 0.1%-1.0%) (ARR, 2.1% [95% CI, 1.0%-3.1%]) for patients with a high RoPE score (Figure 2B). The results were consistent in stability analyses excluding the PC Trial or employing a 9-point (neuroimaging-free) RoPE score (Appendix Figure 3), and with the lead secondary outcome of recurrent ischemic stroke, TIA, or vascular death (Appendix Figure 4). Interaction with age was not significant (P = .9). In the sensitivity analyses that included only the 5 trials that predominantly tested the double-disk device class, the interaction effects were as strong as in the primary analyses (Appendix Figure 5).

Figure 2

Recurrent Ischemic Stroke HTE Analyses for RoPE and PASCAL.

Subgroup analyses based on PASCAL grading showed strong effect modification across the 3 levels of the PASCAL classification on the relative scale; patients with PASCAL classification as unlikely, possible, and probable had HRs of 1.14 (95% CI, 0.53-2.46), 0.38 (95% CI, 0.22-0.65), and 0.10 (95% CI, 0.03-0.35), respectively (P = .003) (Figure 2A), and clinically meaningful differences on the absolute scale at 2 years (Figure 2B). The rates for absolute risk of stroke in the first 2 years for patients with PASCAL classification of unlikely were 3.4% (95% CI, 1.1%-5.7%) and 4.1% (95% CI, 1.7%-6.4%) for the medical therapy and device groups, respectively (ARR, −0.7% [95% CI, −4.0% to 2.6%]). For patients classified as PASCAL possible, the absolute 2-year risks of ischemic stroke was 3.6% (95% CI, 2.4%-4.9%) and 1.5% (95% CI, 0.7%-2.3%) for the medical therapy and device groups, respectively (ARR, 2.1% [95% CI, 0.6%-3.6%]); for patients classified as PASCAL probable, the rates for 2-year stroke risk were 2.5% (95% CI, 1.3%-3.7%) and 0.3% (95% CI, −0.1% to 0.8%) for the medical therapy and device groups, respectively (ARR, 2.1% [95% CI, 0.9%-3.4%]). Again, the results were consistent in stability analyses excluding the PC Trial or employing a 9-point (neuroimaging-free) RoPE score (Appendix Figure 3), and with the lead secondary outcome of recurrent ischemic stroke, TIA, or vascular death (Appendix Figure 4).

The difference in the rates of safety outcomes between the device and medical groups was also consistently higher in the PASCAL unlikely group than in the probable or possible groups (Table 5). For example, the absolute risk increases of postperiprocedural (occurring greater than 45 days after randomization) AF with device were 4.41% (95% CI, 1.02%-7.80%), 1.53% (95% CI, 0.33%-2.72%), and 0.65% (95% CI, −0.41% to 1.71%) in the unlikely, possible, and probable PASCAL categories, respectively, over the full follow-up period. For comparability with 2-year ARR in ischemic stroke, 2-year differences in AF were calculated (Appendix Table 17).

Table 5

Safety Outcomes by PASCAL Classification.

Exploratory Subgroup Analysis

Exploratory single-parameter subgroup analyses (Figure 3) showed nominally stronger RRRs in strata defined by variables postulated to be associated with PFO-related stroke mechanisms (ie, in the theory-anticipated direction), but evidence of effect modification was generally modest. Exploratory subgroup analyses based on the main secondary outcome showed largely similar patterns (Appendix Figure 6).

Figure 3

Recurrent Ischemic Stroke Exploratory Subgroup Analyses.

Technical Efficacy Analysis

The presence of residual shunting (ie, incomplete closure) was assessed in 1475 patients who received the device (287 patients had missing values for residual shunt status). Complete closure was observed in 89.9% of the assessed patients. The primary outcome of recurrent ischemic stroke occurred in 30 (2.3% [95% CI, 1.5%-3.2%]) patients with complete closure, compared with 4 (2.7% [95% CI, 0.7%-6.7%]) patients with incomplete closure (P = .74). The secondary end point of recurrent ischemic stroke, TIA, or vascular death occurred in 66 (5.0% [95% CI, 3.9%-6.3%]) patients with complete closure compared with 9 (6.0% [95% CI, 2.8%-11.2%]) patients with incomplete closure (P = .58).

RoPE Attributable Fraction Analysis

Using a rate of 25% as the expected prevalence of PFO in the general population, we calculated the attributable fraction (ie, the proportion of patients in which the PFO is causally related to the index event rather than just an incidental finding) across the range of RoPE (radiology) 10-point scores. The correlation between the estimated RRR for the primary outcome and the attributable fraction was 0.997 (Figure 4; P < 0.001). A similar correlation was observed with the nonradiology RoPE 9-point score.

Figure 4

Correlation Between Estimated RRR for Primary Outcome and Attributable Fraction.

Concentration of Benefit

The concentration of benefit was assessed across categories of the PASCAL score. The decrease in the 2-year stroke rate (ARR) for possible and probable PASCAL categories was similar, at 2.1%, whereas in the unlikely group, there was observed harm of 0.7%. By treating only patients in the probable group (37%), we estimate that 46% of 2-year events could be avoided. By treating patients in both the possible and probable groups (85%), we estimate that 106% of events could be avoided (since the potential harm of treating the unlikely group is avoided); these results correspond to a Cb of 20%. In leaving out the CLOSURE trial by treating only patients in the probable group (41%), we estimate that 41% of 2-year events could be avoided. By treating patients in both the possible and probable groups (88%), we estimate that 100% of events could be avoided (Cb, 12%).

We also assessed the concentration of benefit across the dichotomized radiology RoPE 10-point score (<7 vs ≥7 points). The decrease in the 2-year event rate for patients with a RoPE score of at least 7 was 2.1%. We estimate by treating only patients in this group (63%) that 76% of 2-year events could be avoided (Cb, 18%). In leaving out the CLOSURE trial, we estimate by treating only patients in RoPE ≥7 group (64%) that 65% of 2-year events could be avoided (Cb, 2%).

Net Benefit of Treatment Selection

We assessed the net benefit of using model-based strategies compared with population-wide strategies of treat all and treat none for patients across a range of decision thresholds. Decision thresholds, corresponding to the number willing to treat to prevent 1 event from 0.5 to 5% (NWT, 200-20) were chosen for measuring the net benefit for the decrease in the 2-year event rate. Among the 5 trials that predominantly tested the double-disk device class, the net benefit of using PASCAL was always greater than both population-wide strategies (Figure 5). Similarly, the radiology RoPE 10-point score was never worse than (and slightly better at high decision thresholds) either population-wide strategy. Net benefit results for the model-based strategies using all 6 trials were similar.

Figure 5

Net Benefit of Treatment Selection Using 5 Trials That Tested the Double-Disk Device Class Across Decision Thresholds.

Summary of Results

This IPDMA indicates that PFO closure in patients with otherwise cryptogenic stroke was associated with a strong RRR for recurrent stroke. The annualized risk of a future stroke for patients assigned to medical therapy was approximately 1%, which accumulates over time; this risk was reduced by device closure by approximately 60%. The benefits associated with device closure were slightly larger when analysis was confined to the 5 trials testing double-disk class closure devices currently used in clinical practice. Overall, PFO closure appeared to be relatively safe. Atrial fibrillation was somewhat more frequent with device closure. However, most AF was transient and did not cause any permanent harm; postperiprocedural AF was increased by only slightly more than 1% on the absolute scale compared with medical therapy.

Subgroup Analyses and HTE

Although the benefits associated with device closure were robust on average, treatment effects varied substantially across strata classified by the probability that the index stroke was PFO related. Device closure did not appear to be associated with any benefit for the 15% of PASCAL unlikely patients who lacked a high-risk PFO (either ASA or large shunt) and had vascular risk factors (ie, RoPE score <7), even with the exclusion of patients with defined stroke mechanisms from these trials. Conversely, device closure was associated with approximately a 90% RRR for patients with a PASCAL probable grade, who had both high-risk PFO characteristics and a high RoPE score. Device closure was associated with intermediate relative effects in the PASCAL possible category. Although relative-effect estimates differed between the probable and possible PASCAL subgroups, the 2-year absolute risk difference among patients in these patients was approximately 2%, for a 2-year number needed to treat of approximately 50, a comparable effect magnitude to that of the 3 mainstays of medical therapy for secondary stroke prevention: antihypertensive therapy, antiplatelet therapy, and statin therapy.

Moreover, the patients who were likely to receive greater benefit also appeared to be at lower risk for device-associated adverse events such as AF, making the harm-benefit trade-offs of device closure more clearly favorable in the possible and probable groups. The lower risk of adverse events in the patients with potential high benefit is consonant with evidence showing a higher risk of incident AF in patients with lower RoPE scores,²⁶ who are older, and who have more vascular risk factors. This increased risk may reflect occult AF being a more likely mechanism for the index stroke in these patients and reflect a greater susceptibility to arrhythmogenic effects of device-tissue contact.

The results illustrate several core concepts of HTE analyses. Even in this pooled analysis of 6 clinical trials, most of the conventional 1-variable-at-a-time subgroup analysis did not generally have estimated effects that would be considered clinically or statistically significant, even though estimated effects were consistently in the theorized direction (eg, attenuated effects for each subgroup with a vascular risk factor). Combining variables creates greater contrast in effects between patients for whom treatment is favorable or not and could help personalize decision-making by more comprehensively describing individuals. The clinical reasoning incorporated into the PASCAL classification system was developed over decades, including the RoPE score, which was derived on an observational database independent from this study.

Lessons Learned

These results also underscore the importance of performing HTE analyses on both relative and absolute scales. Clinically important HTE is variation in the absolute risk difference that spans a clinically important decision threshold,^19,20,64 such as the difference in the treatment effect observed in the PASCAL unlikely strata (where results were null) vs that observed in other strata. Variation on the relative scale may provide mechanistic information, but even strong and statistically significant interaction effects may be clinically unimportant if all groups benefit (as with the RoPE score, when used alone).

Generalizability

Individual participant data meta-analysis has several advantages over study-level meta-analysis,³⁴ including the standardization of analyses across studies, better handling of missing data, the ability to estimate both absolute effects at various time points and conditional treatment effects,^6,12,14 and the opportunity to assess HTE. This study also used a novel approach to predict the patients who are most likely to respond using multiple variables combined, providing new information to inform decisions about which patients should be treated with PFO closure.

Study Limitations

This study has several limitations. First, the magnitude of the benefit associated with device closure with respect to preventing disabling stroke remains uncertain, because substantial data about functional outcomes associated with recurrent stroke were missing. Second, definitions of several key variables, such as large shunt or the presence of an ASA, differed across trials. Nevertheless, this variation across trials did not preclude robust results from pooling data across studies. Third, although the analysis validates both the RoPE score as an indicator of the stroke mechanism (ie, attributable fraction) and the assessed PASCAL system, the original, extended PASCAL classification (Appendix A3, Appendix Figure 1) could not be evaluated because the study did not identify patients in the uncommon extremely high-risk categories (eg, thrombus straddling PFO; patients with concomitant deep vein thrombosis). Fourth, despite the comprehensiveness of this analysis, several important clinical questions remain unaddressed, including the best antithrombotic therapy (eg, anticoagulation vs antiplatelet therapy) with or without closure, the role of new PFO devices, and the role of closure for patients over age 60 years. Fifth, the racial composition was nonrepresentative of US populations. Although race data were collected in only 2 trials (CLOSURE [89.3% White] and REDUCE [92.6% White]), 2 of the remaining databases were European (presumably almost exclusively White) and 1 was South Korean (almost exclusively Asian). Thus, African American patients are underrepresented in this IPDMA, which may influence the generalizability of the results.

Results in Context

To our knowledge, this IPDMA includes the totality of randomized evidence comparing the broad strategy of PFO-closure with medical therapy for patients with PFO-associated cerebral ischemic events. The analyses described provide the most definitive evidence, to our knowledge, to guide management decisions regarding selection of device closure vs medical therapy, addressing key scientific controversies and supporting patient-centered decision-making for patients with PFO-associated cerebral ischemic events. This collaboration and data set also form an ongoing resource to address additional questions related to PFO-associated cerebral ischemic events not directly addressed in the current aims.

Future Research

Although this study focused on understanding who benefits from device closure vs medical therapy, the pooling of these trials can also support future research comparing different medical therapies for PFO-associated stroke (in particular anticoagulation vs antiplatelet therapy)⁶⁴ as well as the potential differential effects and risks of different devices (in particular, double-disk vs umbrella-clamshell devices).⁶

Additional questions for future study include whether selected patients older than 60 years might benefit from closure (including those with high-risk PFO anatomical features/favorable PASCAL classifications) and the potential benefits of new closure devices, including bioabsorbable devices and suture devices.

Appendix A.

Supplementary Methods (PDF, 383K)

Appendix B.

Supplementary Results (PDF, 1.2M)