Methods

Topic Refinement and Review Protocol

Our Evidence-based Practice Center (EPC) developed this topic and Key Questions (KQs) originally through a public process. AHRQ staff refined the topic further for this update of our previously conducted review.

We drafted a protocol for the update of the systematic review. The final protocol can be found on the Effective Health Care Web site (https://effectivehealthcare.ahrq.gov/topics/ptsd-adult-treatment-update/research-protocol); and registered on PROSPERO (Registration number: CRD42017075672).

Literature Search Strategy

Inclusion and Exclusion Criteria

We specified and refined our inclusion and exclusion criteria based on the populations, interventions, comparators, outcomes, timing, and settings (PICOTS) identified for this updated review (Table 2).

Table 2

Inclusion/exclusion criteria for psychological and pharmacological treatments for adults with posttraumatic stress disorder.

Study Selection

Two trained research team members independently reviewed all titles and abstracts identified through searches for eligibility against our inclusion and exclusion criteria. Studies marked for possible inclusion by either reviewer underwent a dual, independent full-text review. For studies without adequate information to determine inclusion or exclusion, we retrieved the full text and then made the determination. We also reevaluated all articles included in the prior PTSD systematic review for inclusion based on the inclusion and exclusion criteria of this updated review. We tracked all results in an EndNote® bibliographic database (Thomson Reuters, New York, NY).

We retrieved and reviewed the full text of all articles included during the title and abstract review phase. Two trained team members independently reviewed each full-text article for inclusion or exclusion based on the eligibility criteria described above. If both reviewers agreed that a study did not meet the eligibility criteria, we excluded the study. If the reviewers disagreed, conflicts were resolved by discussion and consensus or by consulting a third member of the review team. All results were tracked in an EndNote database.

We also recorded the main reason that each excluded full-text publication did not satisfy the updated eligibility criteria (Appendix D). This permitted us to compile a comprehensive list of such studies.

Data Extraction

We pretested abstraction forms and trained abstractors on a set of 10 studies. For studies that met our inclusion criteria, we abstracted relevant information into evidence tables. We designed data abstraction forms to gather pertinent information from each article, including characteristics of study populations, settings, interventions, comparators, study designs, methods, and results. Trained reviewers extracted the relevant data from each included article into the evidence tables. A second member of the team reviewed all data abstractions for completeness and accuracy.

Quality (Risk of Bias) Assessment of Individual Studies

To assess the risk of bias of RCTs, we used the same criteria applied in the 2013 AHRQ review by Jonas and colleagues,⁸⁹ based on the AHRQ Methods Guide for Comparative Effectiveness Reviews.¹⁰⁹ We added a question to the prior risk of bias assessment tool to indicate whether authors reported all prespecified outcomes. The ROBINS-1¹¹² tool (for observational studies) and the Cochrane RCT tool¹¹³ (for RCTs) contain similar criteria as those used for this updated review.

For both RCTs and observational studies, in addition to questions addressing treatment fidelity and whether authors reported all prespecified outcomes, our evaluation included questions to determine selection bias, confounding, performance bias, detection bias, and attrition bias. Concepts covered included adequacy of randomization (for RCTs only), similarity of groups at baseline, masking, attrition, indication of whether authors used intention-to-treat analysis, methods for handling dropouts and missing data, treatment fidelity, and validity and reliability of outcome measures.¹¹⁰ Appendix E shows risk of bias assessments for each study that met inclusion criteria.

Two independent reviewers randomly sampled 10 studies in the 2013 review and reassessed risk of bias using the full set of questions that included the new item assessing reporting of all prespecified outcomes. Because ratings were consistent between original and re-reviewers, we accepted the ratings of each study included in the prior review and rated risk of bias only of newly identified studies. Two independent reviewers assigned these ratings for each new study, and they resolved any disagreements by discussion and consensus or by consulting a third member of the team.

A study rated as “low” has the least bias, and its results are considered valid. A “medium” rating indicates the study has susceptibility to some bias but probably not sufficient to invalidate its results. A study rated as “high” risk of bias has significant bias (stemming from, e.g., serious errors in design or analysis) that may invalidate its results. In general, we gave a low risk of bias rating to studies that met all criteria. Medium ratings were given to studies that presumably fulfilled at least some quality criteria but did not report their methods sufficiently to answer all our questions. We gave a rating of high to studies that had one or more fatal flaws (defined as a methodological shortcoming that leads to a very high risk of bias) in one or more categories. We did not include these studies in qualitative or quantitative analyses used to determine the findings of our review. Appendix F lists each study rated high risk of bias that met the inclusion criteria, along with details on the consistency between the findings from the studies rated as having high risk of bias with those from studies rated as having low or medium risk of bias for each intervention, comparator, and outcome combination reported.

Data Synthesis

We summarized study findings in narrative form; we also created summary tables that tabulate the important features of the study populations, design, intervention, outcomes, settings, and results. All new qualitative and quantitative analyses synthesize studies from the 2013 systematic review that continued to meet criteria of the updated review with those newly identified as a single body of evidence. Appendix G contains evidence tables for all included studies rated low or medium risk of bias.

We carefully considered each body of evidence to determine if findings could be quantitatively pooled as per recent guidance.¹¹⁰ If we found three or more studies with low heterogeneity for a comparison of interest, we performed meta-analysis of the common outcomes from those studies using the metan procedure in Stata and a Dersimonian-Laird estimator based on recent recommendations.¹¹⁴ We also conducted network meta-analysis using the network procedure in Stata¹¹⁵ to compare the pharmacological interventions with each other when we identified at least three studies with low heterogeneity that tested the same intervention with a common comparator (e.g., inactive comparator or active comparator naming a specific intervention) after examining the studies entered as inputs to the network meta-analysis for transitivity. For all analyses, we used random-effects models to estimate pooled or comparative effects.¹¹⁰ For continuous outcomes, we report the standardized mean difference in pre- to postintervention scores between groups when analyses included studies with the same outcome assessments and standardized mean differences when pooling two or more assessment tools. For dichotomous outcomes such as remission and loss of PTSD diagnosis, we report the risk difference between groups. If studies were determined to have moderate heterogeneity, we required five studies to perform meta-analysis of the common outcomes. We did not conduct a network meta-analysis for the psychological interventions because of great levels of heterogeneity in the interventions tested and, to a lesser extent, the comparators used in these studies.

In general, to determine whether quantitative analyses were appropriate, we assessed the clinical and methodological heterogeneity of the studies under consideration following established guidance.¹¹⁰ When bodies of evidence consisted of three studies with low heterogeneity that tested the efficacy or comparative effectiveness of the same intervention (and same comparator when examining comparative effectiveness studies), quantitative meta-analysis was performed when the clinical and methodological heterogeneity of the studies were determined to be insignificant. Similarly, meta-analyses were performed when at least five studies of moderate clinical and methodological heterogeneity tested the efficacy or comparative effectiveness of the same intervention (and same comparator when examining comparative effectiveness studies). The judgment of heterogeneity was done by qualitatively assessing the PICOTS of the included studies, looking for similarities and differences. When we could conduct quantitative syntheses (i.e., meta-analysis), we assessed statistical heterogeneity in effects between studies by calculating the chi-squared statistic and the I² statistic (the proportion of variation in study estimates attributable to heterogeneity). The importance of the observed value of I² depended on the magnitude and direction of effects.

If we include any meta-analyses with considerable statistical heterogeneity in this report, we provide an explanation for doing so, considering the magnitude and direction of effects. When quantitative analyses are not appropriate (because of, e.g., heterogeneity, insufficient numbers of similar studies, or insufficiency or variation in outcome reporting), we synthesize data qualitatively.

Forest plots depict the findings of all quantitative meta-analyses (bodies of evidence with five or more studies or three or four studies with low heterogeneity testing the same intervention). In some instances, when the number of studies was low or the level of heterogeneity for bodies of evidence with three or four studies was high, forest plots are presented without displaying a pooled estimate.

Strength of the Body of Evidence

We graded the strength of evidence (SOE) based on the guidance established for AHRQ’s EPC Program.^{116, 117} Senior members of the review team (including at least one subject matter expert and one methodologist) graded the SOE.

Developed to grade the overall strength of a body of evidence, this approach incorporates five key domains: risk of bias (includes study design and aggregate quality), consistency, directness, precision of the evidence, and reporting bias. It also considers other optional domains that may be relevant for some scenarios, such as a dose-response association, plausible confounding that would decrease the observed effect, and strength of association (magnitude of effect).

Table 3 describes SOE grades. Grades reflect the strength of the body of evidence to answer KQs on the comparative effectiveness, efficacy, and harms of the interventions in this review. Two reviewers assessed each domain for each key outcome; they resolved any differences by consensus discussion. We graded the SOE for all outcomes of interest. Primary outcomes directly associated with PTSD symptoms and associated impairment included decrease in PTSD symptoms, remission (decrease in symptoms below an author-set threshold), and loss of diagnosis (which required the loss of specific number and types of symptoms, timing of symptoms, and loss of associated impairment requisite for a PTSD diagnosis). Other outcomes not directly associated with PTSD symptoms or associated impairment, but still important to overall functioning, included comorbid depression, anxiety, and substance use symptoms; quality of life; disability or functional impairment; and adverse events.

Table 3

Definitions of the grades of overall strength of evidence.

Appendix I displays SOE tables for each study.

Applicability

We assessed the applicability of individual studies, as well as the applicability of the body of evidence following guidance from the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.¹¹⁰ For individual studies, we examined conditions that may have limited applicability based on the PICOTS structure, such as age of enrolled population, index type of trauma experienced (including military-related or combat-related trauma), severity of trauma, and setting of enrolled populations (e.g., hospital, community health center). We also considered whether findings of intervention studies that used Diagnostic and Statistical Manual of Mental Disorders, 4^th edition criteria for PTSD could be extended to individuals meeting Diagnostic and Statistical Manual of Mental Disorders, fifth edition criteria for PTSD.

Peer Review and Public Commentary

Experts in PTSD treatment were invited to provide external peer review of the draft systematic review. AHRQ staff, Patient-Centered Outcomes Research Institute staff, and an Associate Editor reviewed the draft systematic review before it went out for peer review. The EPC Associate Editors are leaders in their respective fields and are actively involved as directors or leaders at their EPCs. Their role is to assess adherence to established methodology and guidelines for EPC-based research. The draft report was posted on the AHRQ Web site from November 15, 2017, to December 29, 2017, to elicit public comment. We revised the report in response to reviewer comments, expanded the analysis strategies, and noted any resulting revisions to the text in the “Disposition of Comments Report.” This disposition report will be made available 3 months after the final systematic review is posted on the AHRQ Web site. Additional details about the expert guidance and review are provided in Appendix J.