NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Treadwell J, Mitchell M, Eatmon K, et al. Imaging Tests for the Diagnosis and Staging of Pancreatic Adenocarcinoma [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2014 Sep. (Comparative Effectiveness Review, No. 141.)
Imaging Tests for the Diagnosis and Staging of Pancreatic Adenocarcinoma [Internet].
Show detailsAnalyses of Comparative Accuracy
Table D-1Summary of analyses of comparative accuracy
| Comparison | Clinical Decision | # Studies | Measure | Test 1 Estimate and 95% CIa | Test 2 Estimate and 95% CIa | Logit Difference and 95% CIb | Statistically Significantly Different? | Precise Enough to Indicate Approximately Equivalent Accuracy? |
|---|---|---|---|---|---|---|---|---|
| MDCT angiography without 3D reconstruction vs. with 3D reconstruction | Resectability in those not staged | 1 | Sensitivity | 89% (95% CI: 68% to 97%) | 100% (95% CI: 83% to 100%) | −1.5 (−4.3 to 1.2) | No | NA |
| MDCT angiography without 3D reconstruction vs. with 3D reconstruction | Resectability in those not staged | 1 | Specificity | 79% (95% CI: 64% to 89%) | 100% (95% CI: 91% to 100%) | −3 (−5.5 to −0.5) | Yes | See above cell |
| MDCT vs. EUS-FNA | Diagnosis | 3 | Sensitivity | 87% (95% CI: 82% to 91%) | 89% (95% CI: 85% to 93%) | −0.2 (−0.8 to 0.4) | No | No |
| MDCT vs. EUS-FNA | Diagnosis | 3 | Specificity | 67% (95% CI: 53% to 78%) | 81% (95% CI: 68% to 90%) | −0.7 (−1.7 to 0.2) | No | See above cell |
| MDCT vs. MRI | Diagnosis | 7 | Sensitivity | 89% (95% CI: 82% to 94%) | 89% (95% CI: 81% to 94%) | −0.01 (−1.4 to 1.5) | No | Yes |
| MDCT vs. MRI | Diagnosis | 7 | Specificity | 90% (95% CI: 80% to 95%) | 89% (95% CI: 74% to 95%) | 0.1 (−2.5 to 2.8) | No | See above cell |
| MDCT vs. PET/CT | Diagnosis | 6 | Sensitivity | 85% (95% CI: 80% to 90%) | 91% (95% CI: 85% to 94%) | −0.6 (−1.2 to 0.1) | No | NA |
| MDCT vs. PET/CT | Diagnosis | 6 | Specificity | 55% (95% CI: 44% to 66%) | 72% (95% CI: 61% to 81%) | −0.7 (−1.4 to −0.1) | Yes | See above cell |
| EUS-FNA vs. PET/CT | Diagnosis | 1 | Sensitivity | 81% (95% CI: 62% to 91%) | 89% (95% CI: 72% to 96%) | −0.6 (−2.1 to 0.8) | No | No |
| EUS-FNA vs. PET/CT | Diagnosis | 1 | Specificity | 84% (95% CI: 62% to 94%) | 74% (95% CI: 51% to 88%) | 0.6 (−0.9 to 2.2) | No | See above cell |
| MRI vs. PET/CT | Diagnosis | 1 | Sensitivity | 85% (95% CI: 64% to 95%) | 85% (95% CI: 64% to 95%) | 0 (−1.6 to 1.6) | No | No |
| MRI vs. PET/CT | Diagnosis | 1 | Specificity | 72% (95% CI: 49% to 87%) | 94% (95% CI: 74% to 99%) | −1.9 (−3.8 to 0.1) | No | See above cell |
| MDCT vs. EUS-FNA | Resectability in those not staged | 1 | Sensitivity | 64% (95% CI: 46% to 79%) | 68% (95% CI: 49% to 82%) | −0.2 (−1.2 to 0.9) | No | Yes |
| MDCT vs. EUS-FNA | Resectability in those not staged | 1 | Specificity | 92% (95% CI: 75% to 98%) | 88% (95% CI: 70% to 96%) | 0.4 (−1.3 to 2.2) | No | See above cell |
| MDCT vs. MRI | Resectability in those not staged | 2 | Sensitivity | 68% (95% CI: 47% to 85%) | 52% (95% CI: 31% to 72%) | 0.7 (−0.6 to 1.9) | No | No |
| MDCT vs. MRI | Resectability in those not staged | 2 | Specificity | 89% (95% CI: 77% to 96%) | 91% (95% CI: 80% to 97%) | −0.2 (−1.7 to 1.2) | No | See above cell |
| MDCT vs. EUS-FNA | T staging | 1 | T staging | Accurate T stage in 41% (95% CI: 20/49); overstaged T in 14% (95% CI: 7/49), understaged T in 44% (95% CI: 22/49) | Accurate T stage in 67% (95% CI: 33/49); overstaged T in 18% (95% CI: 9/49), understaged T in 14% (95% CI: 7/49) | RR 0.61 (0.41 to 0.90) | Yes | NA |
| MDCT vs. EUS-FNA | Vessel involvement | 1 | Sensitivity | 56% (95% CI: 34% to 75%) | 61% (95% CI: 39% to 80%) | −0.2 (−1.5 to 1) | No | No |
| MDCT vs. EUS-FNA | Vessel involvement | 1 | Specificity | 94% (95% CI: 80% to 98%) | 91% (95% CI: 76% to 97%) | 0.4 (−1.3 to 2.1) | No | See above cell |
| MDCT vs. MRI | T staging | 1 | T staging | Accurate T stage in 73% (95% CI: CI 62% to 84%), overstaging in 2% (95% CI: CI 0%–6%), and understaging in 25% (95% CI: CI 14%–36%). | Accurate T stage in 62% (95% CI: CI 49% to 75%), overstaging in 6% (95% CI: CI 0%–12%), and understaging in 32% (95% CI: CI 19%–45%). | RR 1.17 (0.90 to 1.52) | No | No |
| MDCT vs. MRI | N staging | 1 | Sensitivity | 38% (95% CI: 21% to 57%) | 15% (95% CI: 5% to 36%) | 1.2 (−0.2 to 2.6) | No | No |
| MDCT vs. MRI | N staging | 1 | Specificity | 79% (95% CI: 63% to 90%) | 93% (95% CI: 78% to 98%) | −1.3 (−2.8 to 0.2) | No | See above cell |
| MDCT vs. MRI | Metastases | 5 | Sensitivity | 48% (95% CI: 31% to 66%) | 50% (95% CI: 19% to 82%) | −0.09 (−1.2 to 1.0) | No | No |
| MDCT vs. MRI | Metastases | 5 | Specificity | 90% (95% CI: 81% to 95%) | 95% (95% CI: 91% to 98%) | −0.9 (−2.2 to 0.9) | No | See above cell |
| MDCT vs. MRI | Precise stage | 1 | Precise stage | Accurate TNM stage in 46% (95% CI: CI 33% to 59%), overstaging in 8% (95% CI: CI 1%–15%), and understaging in 46% (95% CI: CI 33%–59%). | Accurate TNM stage in 36% (95% CI: CI 23% to 49%), overstaging in 7% (95% CI: CI 0%–14%), and understaging in 57% (95% CI: CI 44%–70%). | RR 1.28 (0.81 to 2.01) | No | No |
| MDCT vs. MRI | Vessel involvement | 2 | Sensitivity | 68% (95% CI: 55% to 79%) | 62% (95% CI: 48% to 74%) | 0.3 (−0.5 to 1.1) | No | Yes |
| MDCT vs. MRI | Vessel involvement | 2 | Specificity | 97% (95% CI: 94% to 98%) | 96% (95% CI: 93% to 98%) | 0.3 (−0.6 to 1.2) | No | See above cell |
| MDCT vs. MRI | Resectability in those staged | 1 | Sensitivity | 67% (95% CI: 48% to 81%) | 57% (95% CI: 37% to 74%) | 0.4 (−0.7 to 1.5) | No | No |
| MDCT vs. MRI | Resectability in those staged | 1 | Specificity | 97% (95% CI: 84% to 99%) | 90% (95% CI: 74% to 96%) | 1.2 (−0.8 to 3.2) | No | See above cell |
| MDCT vs. PET/CT | N staging | 1 | Sensitivity | 26% (95% CI: 14% to 43%) | 32% (95% CI: 19% to 50%) | −0.3 (−1.4 to 0.8) | No | Yes |
| MDCT vs. PET/CT | N staging | 1 | Specificity | 75% (95% CI: 50% to 90%) | 75% (95% CI: 50% to 90%) | 0 (−1.5 to 1.5) | No | See above cell |
| MDCT vs. PET/CT | Metastases | 2 | Sensitivity | 57% (95% CI: 37% to 75%) | 67% (95% CI: 47% to 83%) | −0.4 (−1.6 to 0.8) | No | NA |
| MDCT vs. PET/CT | Metastases | 2 | Specificity | 91% (95% CI: 81% to 97%) | 100% (95% CI: 95% to 100%) | −2.3 (−4.5 to −0.1) | Yes | See above cell |
| EUS-FNA vs. MRI | Precise stage | 1 | Precise stage | Accurate stage for 34/48 patients who had undergone surgical exploration. Of the 34, 34 were stage 2 and below, and 0 was stage 3 or above. The test understaged 13/48, and overstaged 1/48. | Accurate stage for 36/48 patients who had undergone surgical exploration. Of the 36, 35 were stage 2 and below, and 1 was stage 3 or above. The test understaged 12/48, and overstaged 0/48. | RR 0.94 (0.74 to 1.21) | No | Yes |
| MRI vs. PET/CT | Metastases | 1 | Sensitivity | 57% (95% CI: 25% to 84%) | 86% (95% CI: 48% to 97%) | −1.5 (−3.7 to 0.7) | No | No |
| MRI vs. PET/CT | Metastases | 1 | Specificity | 86% (95% CI: 48% to 97%) | 94% (95% CI: 64% to 100%) | −0.9 (−4 to 2.2) | No | See above cell |
- a
If multiple studies, this is the random-effects summary estimate, but if only one study, this is the single-study estimate
- b
For most rows, this column indicates the results of statistical comparison of the two tests using equation 39 of Trikalinos.17 A positive logit difference favors test 1, and a negative logit difference favors test 2. For rows with RR (relative risk), it is the results of the statistical comparison of the two rates using relative risk; RR>1 favors test 1 and RR<1 favors test 2.
NA=Not applicable since the question of equivalence does not apply when a statistically significant difference exists for either sensitivity or specificity; RR=relative risk
Quality of Systematic Reviews
Modified AMSTAR Instrument131,132 for Systematic Reviews
The eight items in boldface below were required to be answered “Yes” in order for a systematic review to be considered high quality. Otherwise, the review was rated not high quality.
- 1.
Was an a priori design or protocol provided?
- 2.
Was a comprehensive search strategy performed?
- 2a.
Was this strategy appropriate to address the relevant Key Question of the CER?
- 3.
Was a list of included and excluded studies provided?
- 4.
Was the application of inclusion/exclusion criteria unbiased?
- 4a.
Are the inclusion/exclusion criteria appropriate to address the relevant Key Question of the CER?
- 5.
Was there duplicate study selection and data extraction?
- 6.
Were the characteristics of the included studies provided?
- 7.
Was the individual study quality assessed?
- 7a.
Was the method of study quality assessment consistent with that recommended by the Methods Guide?
- 7b.
Was the scientific quality of the individual studies used appropriately in formulating conclusions?
- 8.
Were the methods used to combine the findings of studies appropriate?
- 9.
Was the likelihood of publication bias assessed?
- 10.
Have the authors disclosed conflicts of interest?
Table D-2Quality assessments of systematic reviews
| Study | 1 | 2 | 2a | 3 | 4 | 4a | 5 Sel. | 5 Ext. | 6 | 7 | 7a | 7b | 8 | 9 | 10 | Meets Eight Most Important Criteria (High Quality) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Affolter et al. 20131 | No | Yes | No | No | No | No | Yes | Yes | Yes | No | No | No | Yes | No | Yes | No |
| Chen et a. 20132 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes |
| Hébert-Magee et al. 20133 | No | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | No | No |
| Li et al. 201314 | No | No | Yes | No | Yes | Yes | No | Yes | Yes | No | No | No | Yes | No | Yes | No |
| Madhoun et al. 20134 | No | Yes | Yes | No | Yes | Yes | Yes | No | No | Yes | Yes | No | Yes | No | Yes | Yes |
| Wang et al. 20135 | No | Yes | Yes | No | Yes | Yes | No | Yes | Yes | No | No | No | Yes | Yes | No | No |
| Puli et al. 20136 | No | Yes | Yes | No | Yes | Yes | Yes | Yes | No | Yes | Yes | No | Yes | Yes | No | No |
| Chen et al. 20127 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | No | Yes | Yes |
| Hewitt et al. 20128 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes |
| Wu et al. 20129 | No | No | No | No | Yes | Yes | No | Yes | Yes | No | No | No | Yes | No | Yes | No |
| Wu et al. 201210 | No | No | No | No | Yes | Yes | Yes | Yes | No | No | No | No | No | No | Yes | No |
| Tang et al. 200911 | No | No | No | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | No | Yes | No |
| Zhao et al. 200915 | No | No | No | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | No | Yes | No | Yes | No |
| Hartwig et al. 200812 | No | Yes | Yes | No | Yes | Yes | No | Yes | No | Yes | Yes | No | Yes | Yes | Yes | Yes |
| Bipat et al. 200513 | No | No | No | No | Yes | Yes | Yes | No | Yes | Yes | Yes | No | Yes | Yes | Yes | No |
Quality criteria: 1–A priori design or protocol provided, 2–Comprehensive search performed, 2a–Search strategy appropriate to address key questions of this review, 3–Lists of both included and excluded studies provided, 4–Inclusion/exclusion criteria applied in an unbiased manner, 4a–Inclusion/exclusion criteria appropriate to address key questions of this review, 5 sel.–Study selection done in duplicate, 5 ext.–Data abstraction done in duplicate, 6–Characteristics of individual studies reported in evidence table, 7–Quality of each individual study assessed, 7a–Quality assessment consistent with that recommended by the Methods Guide, 7b–Quality of the individual studies used appropriately in formulating conclusions, 8–Data synthesis methods appropriate, 9–The likelihood of publication bias assessed in a qualitative or quantitative manner, 10–Conflicts of interest disclosed by authors. Questions 2, 2a, 4, 4a, 7, 7a, 8, and 10 were deemed “most important.”
Risk of Bias of Comparative Accuracy Studies
- Did the study enroll all, consecutive, or a random sample of patients?
- Was the study unaffected by spectrum bias (e.g., patients with known status before the study, or patients selected for being difficult to diagnose/stage)?
- Was prior experience with the test (technicians, readers) similar for the two imaging tests being compared in the study?
- Were the imaging tests performed within one month of each other (to avoid the possibility that the patient’s true condition changed between tests)?
- Was knowledge of the other test complementary (either both tests were read with knowledge of the other results, or neither test was read with knowledge of the other)?
- Did the interpreters have the same other information available at the time of interpretation for the two imaging tests (other clinical information, 3rd test results)?
- Was each test’s accuracy measuring using the same reference standard (or a similar proportion of patients who underwent different reference standards such as clinical follow-up and surgical findings)?
- Were readers of both tests of interest blinded to the results of the reference standard (or the reference standard was unknowable until after the tests were read)?
- Were the people determining the reference standard unaware of the diagnostic test results?
We defined LOW risk of bias as a study that has a YES for the six boldfaced items above (#2, and #4-#8). We defined HIGH risk of bias as a study that has a NO (or Not Reported) for these six items. We defined MEDIUM risk of bias a study that meets neither the LOW nor the HIGH criteria.
Table D-3Risk of bias assessments of comparative accuracy studies
| Study | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Comments | Risk of Bias |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Fang et al. 201216 | NR | Y | NR | Y | Y | Y | Y | Y | Y | – | Low |
| Herrmann et al. 201217 | NR | Y | Y | N | Y | Y | Y | Y | NR | – | Moderate |
| Tellez-Avila et al. 201218 | NR | Y | Y | NR | NR | NR | Y | Y | Y | Review of data obtained prospectively - EUS and CT - of pancreas lesion that then went to OR for surgical resection with goal of study being detection of vascular invasion, i.e., status of resectability. Not clearly stated but probably results of all tests to date available to all readers. | Moderate |
| Holzapfel et al. 201119 | Y | Y | NR | Y | Y | Y | Y | Y | NR | Two radiologists looked at each images together and came to consensus. Analysis of MDCT and MRI images were spaced 4 weeks apart to avoid any learning bias | Low |
| Koelblinger et al. 201120 | Y | Y | Y | Y | Y | Y | Y | Y | NR | Reading sessions for CT and MR were separated by at least 8 weeks to minimize recall bias, and images were presented to readers in a different randomized order | Low |
| Motosugi et al. 201121 | NR | Y | N | Y | Y | Y | Y | Y | Y | – | Low |
| Rao et al. 201122 | NR | N | Y | NR | Y | Y | Y | Y | NR | Bias against MRI because patients only had MRI if their case was more difficult (see Discussion section of the article). Small tumors only, which are harder to detect, thus possible spectrum bias. | Moderate |
| Shami et al. 201123 | NR | Y | NR | NR | NR | NR | Y | Y | NR | Radiologists for MRI were blinded to EUS result, but did not report the order of the tests or whether EUS readers were blind to MRI result | Moderate |
| Takakura et al. 201124 | NR | Y | Y | N | Y | Y | Y | Y | Y | Has flowchart for included patients, but doesn’t say consecutive or all | Low |
| Imai et al. 201025 | NR | N | NR | NR | Y | Y | Y | Y | NR | Possible spectrum bias because authors imaged for the presence of a particular kind of mets that is hard to detect (para-aortic lymph node metastasis or PALN) | Moderate |
| Lee et al. 201026 | Y | Y | Y | Y | Y | Y | Y | Y | NR | The interval between reads was 2 weeks to minimize learning bias. | Low |
| Kauhanen et al. 200927 | Y | Y | NR | Y | Y | Y | Y | Y | NR | – | Low |
| Farma et al. 200828 | NR | Y | NR | NR | NR | NR | Y | NR | Y | All patients had a peroperative biopsy performed by percutaneous or endoscopi means. Clinical, radiographic, and pathologic follw-up was evaluated for each patient | Moderate |
| Saif et al. 200829 | NR | Y | NR | Y | Y | Y | Y | Y | NR | – | Low |
| Schick et al. 200830 | Y | Y | NR | Y | NR | NR | Y | Y | NR | – | Moderate |
| Casneuf et al. 200731 | Y | Y | NR | NR | N | Y | Y | Y | Y | One reader for MDCT, and two readers together for PET/CT (one of whom had read the MDCT image at least 2 months earlier, and one who had read the PET alone image 2 at least months earlier). For PET/CT they had to come to consensus. This design is a bias in favor of PET/CT because there were always 4 eyes on the PET/CT, whereas CT only had 2 eyes. In addition, the PET/CT assessment was probably influenced (improved?) by the two readers’ prior memory of the two individual scans. | Moderate |
| Tamm et al. 200732 | NR | Y | NR | NR | N | N | Y | Y | N | MDCT images were read without knowledge of clinical, pathologic, or surgical data, or EUS-FNA findings. EUS-FNA was performed with knowledge of the MDCT finding. Also the reference standard for some patients was determined by the FNA. This design is a bias in favor of EUS-FNA. | Moderate |
| Mehmet Ertuk et al. 200633 | Y | N | NR | Y | Y | Y | Y | Y | Y | Patient statuses were all known beforehand, hence probably spectum bias. MDCT was always read first. The interval between reads was 4 weeks to minimize learning bias. | Moderate |
| Heinrich et al. 200534 | NR | Y | NR | Y | NR | NR | Y | NR | NR | Findings on PET/CT were compared with results of standard staging and validated by intraoperative findings and histology of the resected specimen or biopsies. For patients who were diagnosed to have benign pancreatic lesion by PET/CT and did not undergo resection, long-term outcome was assessed to confirm the diagnosis made by PET/CT | Moderate |
| Agarwal et al. 200435 | Y | Y | NR | NR | NR | Y | Y | Y | Y | – | Moderate |
| DeWitt et al. 200436 | NR | Y | Y | Y | Y | Y | Y | Y | N | Readers for neither test were blind to previous radiographic data. Readers of the 2nd test (which was always MDCT) were blind to results from the 1st (which was always EUS-FNA) | Low |
| Lemke et al. 200437 | NR | Y | NR | Y | N | NR | Y | NR | NR | 2 radiologists evaluated the original CT and PET images as well as the fused images in a randomized order in 3 different settings with an interval of 2 weeks each, using a standardized questionnaire. | Moderate |
| Soriano et al. 200438 | Y | Y | NR | NR | Y | Y | Y | Y | Y | Surgeons only saw a combined report of all the imaging tests, and did not know individual imaging results | Moderate |
| Rieber et al. 200039 | NR | Y | NR | NR | N | Y | Y | Y | NR | Readers: 3 different radiologists blinded to all clinical data regarding the patient. | Moderate |
- Analyses and Risk of Bias Assessments - Imaging Tests for the Diagnosis and Stag...Analyses and Risk of Bias Assessments - Imaging Tests for the Diagnosis and Staging of Pancreatic Adenocarcinoma
Your browsing activity is empty.
Activity recording is turned off.
See more...