Table 5Strengths and Limitations of Clinical Studies Using the Downs and Black checklist26

StrengthsLimitations
Randomized Controlled Trials
Kim et al. (2020)33
  • The study’s objective, intervention, and main outcomes were clearly described
  • Population characteristics were clearly described, and eligibility criteria given
  • The major findings of the study were described in a way that allowed verification of analyses and conclusions
  • Estimates of random variability were reported
  • Data analyses were planned at the outset
  • The time period over which patients were recruited was specified
  • Length of follow up was consistent between the intervention and comparator groups
  • Although the characteristics of the patient included in the study are clearly described for the ITT population, the characteristics of the per protocol population or of the withdrawals and dropouts were not provided.
  • Authors did not describe how missing data were handled
  • This study was open label. This may have introduced observer biases for subjective outcomes
  • The study was multicenter (university hospitals), which may not be representative of the usual primary care setting for patients with type 2 diabetes
  • Although the allocation of patients to the treatments groups was randomized, authors did not indicate their randomization method
  • The main conclusions of the study were based on the analysis of treatment (i.e., per protocol) rather than ITT
  • Authors did not mention personal conflicts of interest
  • Although the study’s funding source (Takeda Pharmaceutical Korea Company) was declared, it is unclear if it may have influenced the editorial independence of the authors
Khaloo et al. (2019)34
  • The study’s objective, intervention, and main outcomes were clearly described
  • Population characteristics were clearly described, and eligibility criteria given
  • The major findings of the study were described in a way that allows verification of analyses and conclusions
  • Estimates of random variability were reported
  • Data analyses were planned at the outset
  • The time period over which patients were recruited was specified
  • Analyses were done according to ITT
  • Missing data were handled using the “last observation carried forward” method
  • There was no characterization of the patients who withdrew
  • There was no mention of any patients lost to follow-up, nor was there any mention of adjusting the analyses for different lengths of follow-ups
  • This study was open label. This may have introduced observer biases for subjective outcomes
  • This was a single center study and may not be representative of treatment available for most of the source population, therefore limiting generalisability
  • Adherence to the intervention was not mentioned, therefore it is unclear if group contamination may have occurred
Non-Randomized Studies
Yen et al. (2020)35
  • The objectives, patient characteristics, interventions, controls, and outcomes were well described
  • Distribution of principal confounders were well balanced due to propensity score matching
  • No loss to follow up due to retrospective cohort design
  • Appropriate statistical analysis
  • Authors declared no conflict of interest
  • Unclear if observers were blinded
  • Actual probability values were not reported
  • 15,218 patients from the analysis set could not be matched (non-PIO, n = 15,169; PIO, n = 49), which could impact generalizability
  • An important confounder missing from propensity score calculations was hemoglobin A1c values which reflects disease severity
Yen et al. (2020)36
  • The objectives, patient characteristics, interventions, controls, and outcomes were well described
  • Distribution of principal confounders were well balanced due to propensity score matching
  • No loss to follow up due to retrospective cohort design
  • Appropriate statistical analysis
  • Authors declared no conflict of interest
  • Unclear if observers were blinded
  • Actual probability values were not reported
  • 86,792 patients from the analysis set could not be matched (never used TZD, n = 69,031; used TZD, n = 17,761), which could impact generalizability
  • An important confounder missing from propensity score calculations was duration of disease
Cid Ruzafa et al. (2019)37
  • The objectives, patient characteristics, interventions, controls, and outcomes were well described
  • No loss to follow up due to retrospective cohort design
  • The source population is provided (entire population from Denmark), and authors adequately describe how participants were selected
  • Authors adjusted their analyses to account for different lengths of follow-up between patients
  • Compliance with the intervention was reliable, decreasing the probability of misclassification
  • Main outcomes measures were valid and reliable (presence of a diagnostic code)
  • Patients in the different intervention and control groups were selected from the same population
  • Main findings are not presented in a way that allows the reader to verify the major analyses and conclusions
  • Although estimates of the random variability in the data are offered for the main outcomes, authors presented interquartile range for normally distributed data (laboratory test results) instead of SD, standard error, or CIs
  • The effect of the main confounders was not investigated nor was adjustments made in the final analyses
  • Authors reported the study’s funding source (i.e., the drug manufacturer) as well as personal conflicts of interest (e.g., employees of the drug manufacturer); however, they did not discuss how these were managed. It is unclear if this may have influenced editorial independence
Miao et al. (2019)38
  • The objective of the study, patient characteristics, interventions, comparators and outcomes were clearly described
  • A list of principal confounders was given and were compared across the groups and adjusted for during the analysis
  • Main findings of the study were clearly described as simple outcome data. Study also provided estimates of random variability using SDs and CIs as appropriate. Actual probability values were reported when the P value was > 0.001
  • No loss to follow up due to retrospective cohort design
  • The statistical analyses conducted were appropriate and predefined
  • Patients in the different intervention and control groups were selected from the same population over the same period of time
  • Potential confounders were adjusted for in the comparative analysis
  • Study authors had no conflict of interest to declare
  • Participants and outcome assessors were not blinded
  • The baseline characteristics of the groups were different for potential confounders
  • An important confounder missing from propensity score calculations was hemoglobin A1c values which reflects disease severity
  • The duration of exposure to the intervention and comparators were not similar
  • No other AEs were measured
Patorno et al. (2019)39
  • The objectives, patient characteristics, interventions, controls, and outcomes were well described
  • Authors reported the study’s funding sources and declared that editorial independence was retained
  • Authors adjusted their analyses for different lengths of follow-up
  • Propensity score matching was utilized to minimize the influence of confounders between groups and no important confounder appeared to be missing from propensity score calculations
  • Main outcome measures used were valid and reliable
  • Patients from the intervention and comparator groups were recruited from the same population and over the same period of time
  • Although the characteristics of the patient included in the study are clearly described, the characteristics of patient withdrawals and dropouts was not provided
  • Although patients were representative of the entire population of their dataset, they may not be representative of the type 2 diabetes population at large. For instance, uninsured individuals would not be represented in this study and this would limit its generalizability.
  • Since this is a database study, the patient’s adherence to the intervention can not be fully ascertained.
  • No statistical adjustments were made for multiple testing over time
  • Authors reported the study’s funding source (e.g., drug manufacturer) as well as personal conflicts of interest (e.g., employees of drug manufacturer); however, they did not discuss how these were managed. It is unclear if this may have influenced editorial independence
  • 236,108 patients initiating LIN or PIO from the analysis set could not be matched, which could impact generalizability.
Spence et al. (2019)40
  • The objective of the study, patient characteristics, interventions, comparators and outcomes were clearly described
  • The primary and secondary outcomes were clearly defined in the methods section
  • Potential confounders (e.g., smoking status, blood pressure, BMI) were listed and compared across treatment and placebo groups.
  • Main findings of the study are clearly described as simple outcome data. Study also provided estimates of random variability using SD and CIs as appropriate
  • Important AEs were reported. Actual probability values were reported when the P value was > 0.001
  • The study was an international multicenter trial and was representative of the population of interest
  • The statistical analyses of the secondary analysis were appropriate and predefined with a statistical analysis plan
  • Adherence was formally assessed using pill counts
  • An ITT analysis was done
  • Post-hoc analysis of a subgroup of patients enrolled in a previous RCT.
  • Patients lost to follow up were not clearly described
  • The treatment of missing data in the ITT analysis was not reported

AE = adverse event; BMI = body mass index; CI = confidence interval; ITT = intention-to-treat; NR = not reported; RCT = randomized controlled trial; SD = standard deviation.

From: Pioglitazone for Type 2 Diabetes Mellitus and Pre-Diabetes: A Review of Safety

Cover of Pioglitazone for Type 2 Diabetes Mellitus and Pre-Diabetes: A Review of Safety
Pioglitazone for Type 2 Diabetes Mellitus and Pre-Diabetes: A Review of Safety [Internet].
Marchand DK, Subramonian A, Featherstone R, et al.
Copyright © 2020 Canadian Agency for Drugs and Technologies in Health.

The copyright and other intellectual property rights in this document are owned by CADTH and its licensors. These rights are protected by the Canadian Copyright Act and other national and international laws and agreements. Users are permitted to make copies of this document for non-commercial purposes only, provided it is not modified when reproduced and appropriate credit is given to CADTH and its licensors.

Except where otherwise noted, this work is distributed under the terms of a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International licence (CC BY-NC-ND), a copy of which is available at http://creativecommons.org/licenses/by-nc-nd/4.0/

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.