An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Jeffery Gerber, Rachel Ross, Matthew Bryan, A. Russel Localio, Julia Szymczak, Alexander Fiks, Darlene Barkman, Folasade Odeniyi, Kathryn Conaboy, Louis Bell, Theoklis Zaoutis, and Richard Wasserman.
Author Information and AffiliationsOutpatient acute respiratory tract infections (ARTIs), including acute otitis media, streptococcal pharyngitis , and acute sinusitis, account for the vast majority of antibiotic exposure in children, and much of it is inappropriate. Broad-spectrum antibiotic prescribing for ARTIs has increased. It remains unclear if treating ARTIs with broad-spectrum antibiotics compared with treatment with narrow-spectrum agents leads to improved outcomes. Primary care providers, patients, and caregivers would benefit from studies assessing the implications of alternate antibiotic regimens for these common infections.
(1) Identify the outcomes most important to parents’ and children regarding the treatment of ARTIs with antibiotics, and (2) compare the effectiveness of narrow- and broadspectrum antibiotic therapy for ARTIs on patient- and clinician-centered outcomes.
The study was performed in a network of 31 pediatric primary care practices. To identify patient-centered outcomes, we conducted semistructured interviews with caregivers of children attending acute office visits and brief interviews with children, which were analyzed by 2 coders using a modified grounded theory approach. To compare the effectiveness of antibiotics by spectrum, we performed 2 studies. First, to assess patient-centered outcomes, we conducted a prospective observational cohort study. Between January 2015 and April 2016, a stratified sample of caregivers of children treated with antibiotics for ARTIs was assessed according to the previously identified patient-centered outcomes from telephone interviews at 5 to 10 days and at 14 to 20 days postdiagnosis. The primary patient-centered outcome was health-related quality of life (QOL) as determined by the Pediatric Quality of Life Inventory (PedsQL) total score (measured at first interview). To assess clinician-centered outcomes, we conducted a retrospective observational cohort study, including all children treated with antibiotics for ARTIs in 2015, using electronic health record data. To control for confounding, 2 analytic approaches—a stratified analysis and a propensity score matched analysis—were implemented.
To achieve the first objective, we interviewed 109 parents and 24 children. Parents identified 4 important outcomes related to antibiotic use for their child’s ARTI: disrupted sleep, missed school and parent work, child suffering, and speed of symptom resolution. Parents confirmed the importance of diarrhea and gastrointestinal distress and having an allergic reaction, outcomes we had previously identified as important. For children, missing school, disrupted sleep, and speed of symptom resolution were outcomes of concern. For the comparative effectiveness objective, narrow-spectrum antibiotics were associated with a higher health-related QOL score (PedsQL score difference: stratified approach 1.6; 95% CI, 0.5-2.8; propensity-score matched approach: 1.4; 95% CI, 0.4-2.4). Compared with narrow-spectrum antibiotics, broad-spectrum antibiotics were not associated with improved outcomes on any of the secondary patient-centered outcomes but were associated with increased rates of adverse drug effects as reported by parents (raw %: narrow 25.2%, broad 35.5%; stratified approach RD [risk difference] = –11.4%; 95% CI, –17.0% to –5.8%; propensity-score approach RD = –11.4%; 95% CI, –16.4 to –6.4).
Treatment with broad-spectrum antibiotics was not better than narrow-spectrum antibiotics for children with ARTIs. Receipt of broad-spectrum antibiotics was associated with higher rates of adverse drug effects. Given current national prescribing patterns, outpatient antimicrobial stewardship efforts should focus on antibiotic selection for the most common childhood ARTIs.
Antibiotics are by far the most common prescription drugs given to children.1 In 2011, children received nearly 74 million outpatient antibiotic prescriptions—almost 1 for every child in the United States2—accounting for roughly a fourth of all prescription medications for this age group. Four of the top 5 most frequently prescribed drugs were antibiotics. Albuterol, the sole nonantibiotic in the top 5, was prescribed 5 times less frequently.1 Therefore, it is imperative that we prioritize research that can direct clinicians to prescribe these medications appropriately.
Although hospitalized children frequently receive antibiotic prescriptions,3 the vast majority of antibiotic use occurs in the outpatient setting, roughly 75% of which is for acute respiratory tract infections (ARTIs).4 ARTIs are the most common reason for sick visits to primary care pediatricians, and up to half of the visits for ARTIs result in an antibiotic prescription.4-6 Three conditions—acute otitis media (AOM), group A streptococcal (GAS) pharyngitis, and acute sinusitis—account for almost all bacterial ARTIs. Thus, focusing comparative effectiveness studies on the management of this small group of diagnoses is an efficient and high-impact strategy for improving the use of antibiotics for children.
Initial research and public health campaigns focused on the prevention of prescribing antibiotics for viral infections (ie, those infections for which antibiotics are not needed). As a result of these efforts, unnecessary prescribing for viral infections has declined.5,7,8 At the same time, however, the rate of broad-spectrum antibiotic prescribing has dramatically increased.9 The overuse of broad-spectrum antibiotics represents another form of inappropriate prescribing. For example, when broad-spectrum antibiotics are used to treat infections for which narrow-spectrum antibiotics are sufficient, the potential for unnecessary bacterial resistance pressure, drug-related adverse effects, and costs could magnify. Although ARTIs are generally caused by pathogens that are susceptible to narrow-spectrum antibiotics, national estimates document that these conditions are increasingly treated with broad-spectrum antibiotics.4,7,9-11 We have identified the same trend across the Children’s Hospital of Philadelphia primary care network, which revealed unexplained variation in antibiotic prescribing, including more than 5-fold differences across practices in the use of broad-spectrum antibiotics for common ARTIs.12 This variability remained despite adjustment for patient age, sex, race, insurance type, recent antibiotic use, and antibiotic allergies, indicating that similar children are receiving different antibiotic choices for the same conditions. These data identify both the potential overuse of broad-spectrum antibiotics and the potential utility of this network to perform large-scale studies.
There have been no large comparative effectiveness studies comparing narrow- and broad-spectrum antibiotic therapy for the most common bacterial ARTIs. Although studies have demonstrated that antibiotics are indicated for these infections, prior studies are based on comparisons between either a narrow- or a broad-spectrum antibiotic and a placebo, rather than head-to-head comparisons between narrow- and broad-spectrum antibiotics. Thus, it is currently unclear whether narrow- or broad-spectrum antibiotics are more appropriate. As a result, there is a lack of adequate information to help parents and clinicians determine how to treat children with common ARTIs.
The variability outlined above likely reflects this lack of data. Specifically, the latest national guidelines and expert recommendations for antibiotic therapy do not agree. For example, the American Academy of Pediatrics (AAP) recommends penicillin or amoxicillin, both narrow-spectrum agents, as first-line therapy for AOM13; however, 2 recent, high-profile randomized controlled trials chose a broad-spectrum agent, amoxicillin-clavulanate, when comparing antibiotic to placebo for the treatment of AOM—a choice implying that this broad-spectrum antibiotic is preferred. 14,15 Likewise, the most recent AAP guidelines recommend amoxicillin, a narrow-spectrum agent, for the treatment of children with acute sinusitis,13 in contrast to practice guidelines from the Infectious Diseases Society of America, which recommend amoxicillin-clavulanate, a broad-spectrum antibiotic. Even the treatment of GAS pharyngitis, a condition for which narrow-spectrum drugs have been historically recommended, has been challenged recently.16 17 These mixed messages likely contribute to the inconsistent management observed in the primary care network, which likely extrapolates to either over- or undertreatment of millions of children each year across the United States. Without improved data, practitioners and families will lack sufficient guidance for making appropriately informed decisions.
The rationale for these emerging recommendations toward broader-spectrum antibiotics is not based on clinical evidence but on the predicted effects of a changing microbiology. Specifically, recent surveillance data of respiratory tract pathogens, associated with the increased uptake and spectrum of pneumococcal vaccination, has resulted in a relative decrease in carriage of amoxicillin-sensitive pathogens (Streptococcus pneumoniae) and a relative increase in the frequency of amoxicillin-resistant pathogens (Haemophilus influenzae, Moraxella catarrhalis). The theoretical clinical consequences of this shift in microbiology, including the need for broader-spectrum empiric antibiotic therapy, however, have not been confirmed. Therefore, real-world clinical studies with patient-centered outcomes must be conducted to compare narrow- and broad-spectrum antibiotics. Organizations producing guidelines, clinicians, and families would collectively benefit from large pediatric comparative effectiveness studies designed to determine how best to use antibiotics for the most common pediatric ARTIs.
Inappropriate prescribing of antibiotics might not be especially problematic for patients if antibiotics had no harmful side effects, but they do. Harmful effects of antibiotic exposure include both drug-related adverse effects and the promotion of antibiotic resistance. Multiple studies have estimated the types and frequencies of adverse effects of antimicrobial use.18-23 Antibiotics are the leading cause of drug-related adverse effects resulting in emergency department visits for children in the United States.24 In fact, rates of antimicrobial-related adverse events were 3 times higher than rates of adverse drug events attributable to the most high-risk medicines, including anticoagulant and antiplatelet agents (eg, aspirin and clopidogrel), oral hypoglycemics (eg, metformin), and narrow therapeutic index agents (eg, phenytoin and lithium). These prior studies are limited, however, by a lack of patient-specific data linking outpatient antimicrobial use to actual adverse events, by incomplete adverse event reporting (adverse events have been captured from 1 setting only, usually the emergency department or hospital, when multiple contact sites for medical care exist), and by a relative lack of pediatric data—limitations that would be mitigated by exploring adverse event rates after antimicrobial use prospectively within a large pediatric population. Determining these outcomes at the patient level would help to inform the shared decision-making process between families and clinicians, as well as national guidelines for antibiotic management of common ARTIs.
In addition to direct patient harm outlined above, the public health implications of inappropriate antibiotic use are profound.25 The emergence of antimicrobial resistance is a major threat to the health of individuals and society.26,27 The Centers for Disease Control and Prevention describes antibiotic resistance as “one of the world’s most pressing health problems” and the World Health Organization has identified antibiotic resistance as “one of the three greatest threats to human health.” Infections with resistant bacteria increase morbidity and mortality, and greatly escalate the cost of medical care. The Institute of Medicine estimated that, in 2010, roughly $20 billion was spent on the treatment of antibiotic-resistant infections. Even worse, the introduction of newer, effective antibiotics has slowed significantly.28 Antibiotic use has been associated with the emergence of antimicrobial resistance,29-31 including for common respiratory pathogens such as GAS30,32-34 and Streptococcus pneumoniae, 35-41 as well as for both narrow- and broad-spectrum antibiotics, including penicillins,35 macrolides,32,35,39,40 and fluoroquinolones.37 Furthermore, broad-spectrum antimicrobials are more expensive than narrow-spectrum agents. For example, for a 10-kg child, a 10-day course of amoxicillin costs roughly $15 compared with $50 for 5 days of azithromycin or 10 days of cefdinir—health care savings that could become substantial given the volume of prescriptions.
Notwithstanding these important economic and public health implications of different approaches to managing ARTIs, decisions regarding antibiotic choice for these common childhood infections should be based on outcomes important to the key stakeholders: the patients and their parents. In pediatrics, considerable research has been conducted to better understand what parents think about antibiotics for their children, because there is a widely held and durable perception on the part of pediatricians that parent demand for antibiotics is a reason why overuse occurs.42 We know that pediatricians may overestimate parent demand for antibiotics and that this misperception drives overuse.43,44 The research to date demonstrates that parents hold many misconceptions about antibiotics and ARTIs45 but that they may be becoming more informed and sophisticated regarding the appropriate use of these drugs.46 We know little about which outcomes related to antibiotic use and the management of ARTIs are important to parents or children. Because previous clinical trials and comparative effectiveness studies have been largely centered on outcomes deemed important to clinicians and researchers, however, the impact of broad- versus narrow-spectrum antibiotics on patient-centered outcomes has not been determined. Thus, to inform a meaningful comparative effectiveness study that will change practice, it is critical to identify and prioritize ARTI outcomes that are important to patients and their caregivers. An additional benefit of this approach is that it allows us to compare the results using patient-centered outcomes with more traditionally used outcomes observed by clinicians.
Outpatient ARTIs account for the vast majority of antibiotic exposure in children, and roughly half of these antibiotic prescriptions are inappropriate.47 Although unnecessary antibiotic prescribing for viral infections has significantly decreased, there has been a substantial increase in prescribing of broad-spectrum antibiotics to treat ARTIs when narrow-spectrum antibiotics are indicated.9 Primary care providers, patients, and caregivers would benefit from studies assessing the implications of alternate antibiotic regimens for these common infections. Specifically, it remains unclear if treating common ARTIs with broad-spectrum antibiotics leads to an improvement in patient outcomes compared with treatment with narrow-spectrum antibiotics, particularly considering the growing threat posed by antimicrobial resistance. Because of the lack of large comparative effectiveness studies with patient-centered outcomes to address this issue, professional guidelines and expert recommendations are conflicting, and, as a result, practice patterns vary considerably. Furthermore, the adverse effects of antibiotic use in children have not been clearly defined or quantified. To address these knowledge gaps, we conducted a patient-centered, comparative effectiveness study to achieve the following 2 aims:
In an attempt to build trust, transparency, respect, and patient partnership in our quest to answer the questions posed by this application, we involved key stakeholders in every aspect of our project. The key stakeholders in this study included (1) children affected by acute respiratory tract infections, (2) their caregivers, and (3) primary care providers (PCPs) who manage their disease. We included each of these groups in the formulation of the research question, defining essential characteristics of the comparators and outcomes, monitoring study conduct and progress, and developing strategies to disseminate and implement the research findings.
In formulating the research question, we consulted with leaders of the Children’s Hospital of Philadelphia (CHOP) Family Advisory Council (FAC), a patient and family advocacy group. Composed of family members whose children are or have been at CHOP, the FAC has a broad representation of families from diverse backgrounds with myriad diagnoses and experiences within the CHOP network. It also includes staff from many disciplines, all committed to advancing family-centered care. The goal of the FAC is to provide the patient and family perspective and to share expertise to support the hospital’s family-centered care objectives. We presented our protocol during a monthly FAC meeting, where we solicited input on the importance of our research question. Kathryn Conaboy and Darlene Barkman, leaders of CHOP’s FAC and co-investigators on this project, led this discussion. We also engaged a broad representation of clinicians in the study team, including 2 pediatric infectious diseases clinicians and multiple general pediatricians, who reviewed the study protocol and instruments and provided feedback on the study design.
The entire first aim of this project was devoted to deriving the comparative effectiveness study outcomes from key stakeholders. To obtain these study outcomes, we partnered with patients and families affected by the targeted conditions as well as the clinicians who treat them. We conducted semistructured interviews with caregivers and patients from a diverse mix of practice sites to gather their impressions about antibiotics and the treatment of common respiratory tract infections. The data obtained from these interviews was central to defining study outcomes of ARTIs treated with antibiotics, the validity and design of the protocol, and the design of our data collection instrument utilized in Aim 2. We also consulted with several PCPs within the network to help refine the study outcome classification scheme for the electronic health record based analysis. These discussions both verified some of our a priori assumptions and generated novel patient-centered outcomes.
Our FAC consultants were instrumental in monitoring study conduct and progress. FAC consultants were present at all team meetings and provided input as co-investigators. For example, before enrollment during both aims of the study, FAC consultants helped monitor and troubleshoot any issues regarding patient recruitment, enrollment, and follow-up. For example, for Aim 2, FAC consultants directed us to modify the protocol early on to administer the guide as 2 interviews to make it more manageable for parent and patient stakeholders. When enrollment was slow early on, the family partners helped strategize ways to increase enrollment while maintaining data integrity and consistency.
In addition to the traditional dissemination modes of scientific conferences and peer-reviewed publication, we have developed a comprehensive dissemination and implementation plan with 5 components: (1) patient partners; (2) the CHOP Care Network and Pediatric Research Consortium; (3) CHOP PolicyLab: the Center to Bridge Research, Practice, and Policy; (4) the Leonard Davis Institute of Health Economics at Penn; and (5) CHOP’s public relations and marketing department. Our research can create change only to the extent to which it is communicated—at the right time, and in the right formats—to our key stakeholders and the larger community. First, to disseminate our findings to patient partners, we will continue to meet and engage with FAC partners and communicate our results to study participants through CHOP’s FAC newsletter “CHOP Family News.” Second, to disseminate our findings to clinicians, we will participate in meetings with CHOP’s Pediatric Research Consortium leadership to strategize our communication methods using the channels deemed most effective by these key stakeholders. The study team will also work with the CHOP clinical pathways program, which aims to standardize care and improve outcomes, to incorporate the results of the study. Lastly, to communicate our findings beyond CHOP, we will work with CHOP’s PolicyLab, the Leonard Davis Institute, and CHOP Public Relations to develop materials that will present our findings in a clear, accessible, and interactive format. As our study team continues to refine analyses, we will continue to consult FAC members and physician stakeholders to elicit their feedback on how to communicate the results effectively to our patient partners and the lay public.
The study was performed in a network of 31 pediatric primary care practices with a common electronic health record (EHR; Epic Systems) serving children of diverse racial and socioeconomic backgrounds across southeastern Pennsylvania, southern New Jersey, and northern Delaware.48 The city of Philadelphia, the location of 4 of the 31 practices, has one of the highest poverty rates in the United States (25.8%).49 The 4 practices in Philadelphia are also academic teaching practices where medical students and pediatric residents receive training. The Children’s Hospital of Philadelphia IRB approved this study.
To identify the outcomes most important to the parents and children regarding the use of antibiotics for ARTI.
To elicit patient- and parent-centered outcomes, we conducted a 2-phase qualitative study. Phase 1 was focused on eliciting parent perceptions about antibiotics at the time their child presents to an ambulatory setting with symptoms of an ARTI. Phase 2 was focused on eliciting child perceptions about antibiotics. We gathered these qualitative data in 2 phases because we were interested in eliciting parent perceptions about antibiotics at the time in which they are caring for a child with symptoms of an ARTI. Based on a review of the literature on best practices for interviewing children50 and suggestions from our family partners, we determined that interviewing children at the time they are symptomatic would lead to incomplete, inaccurate, and inconsistent data. Therefore, we decided to interview otherwise well children separately from the parents. In Phase 1, we conducted semistructured interviews with parents or guardians of children presenting with ARTI symptoms to 1 of 4 diverse practices in the primary care network between March 2014 and February 2015. We selected 2 large urban practices located in Philadelphia and 2 large suburban practices located outside the city from which to recruit interview respondents. Parents were eligible for inclusion if they had a child between 3 months and 18 years who had at least 1 chief complaint consistent with an ARTI (eg, sore throat, ear pain, congestion, fever, or cough). Parents were enrolled over 12 months until saturation of key themes was achieved.51
To recruit interview respondents, a research assistant arrived at each practice in the morning and reviewed the reasons for the day’s scheduled sick visits via the EHR. Parents of children with at least 1 of 26 chief complaints that might be consistent with 1 of the targeted conditions (eg, sore throat, ear pain, fever, cough; Table 1) were considered eligible for inclusion. Once eligible parents were identified, the research assistant notified the front desk staff. When an eligible parent and child checked in for their appointment, front desk staff asked them if they would be interested in being approached about participation in a research study. Interested parents were recruited by the research assistant and, if they agreed to participate, were brought into a private room in the practice before being seen by the doctor. We provided age-appropriate books and toys for the child to play with while the parent was being interviewed.
Chief Complaint Criteria Used to Identify Eligible Respondents.
In Phase 2, interviews were conducted with children between January and March 2015. We recruited children aged 8 and older to participate in 2 ways. First, we invited members of the CHOP Youth Advisory Council (YAC) to participate in interviews via email. The YAC is a group of 30 CHOP patients who meet monthly to provide input on hospital policies. Second, we recruited children from 2 of the practices from which parents were recruited in Phase 1 (1 urban and 1 suburban practice). Interviews with YAC members were conducted over the telephone; interviews with children recruited from the practices occurred in person. Children were enrolled over 3 months until saturation of key themes was achieved.
Before beginning Phase 1 data collection, we created a semistructured interview guide based on a review of the literature and discussions among the research team. The interview guide was piloted with members of the Children’s Hospital of Philadelphia Family Advisory Council and amended for length and comprehensibility of concepts. The interview guide included 2 sections (Table 2). The first section included a series of open-ended questions intended to get parents to share their reasons for bringing their child to the doctor, expectations for the visit, and opinions about antibiotics—with minimal prompting. The second section included a series of closed-ended questions assessing parental concern about antibiotic risks, treatment failure, and side effects. The interview concluded with a short demographic survey. The child interview guide for Phase 2 was a modified version of the guide used for parents. We included age-appropriate open-ended question prompts and a series of closed-ended questions that mirrored those we asked of the parents. All interviews were conducted by a research assistant trained in interview techniques by a medical sociologist, and all sessions were audio recorded with the permission of the respondent. Each respondent was asked the same set of questions from the interview guide, with the research assistant probing and redirecting the conversation to elicit more in-depth data or to clarify points as necessary.52 At the conclusion of the interview, the respondent was given a $25 gift card to thank them for participating.
Interview Guide Question Domains.
All interview audio was transcribed and uploaded into NVivo 11 qualitative data analysis software (QSR 2016) for management and analysis. Two analysts independently and systematically analyzed the transcripts using a modified grounded theory approach.53,54 First, the analysts read through all transcripts and identified groups of salient, repeated themes that would serve to define the outcomes for the patient-centered cohort. These themes were defined by consensus and created as codes in NVivo. Second, after the code list was developed and refined, the analysts reviewed all transcripts line by line to determine which codes fit the concepts suggested by the data. Intercoder reliability was assessed periodically throughout analysis, and modifications were made to the coding procedure until reliability exceeded 95%. We analyzed data from the closed-ended questions that consisted of descriptive statistics showing distributions.
We conducted a prospective cohort study to assess patient-centered outcomes after diagnosis and treatment of an ARTI with an antibiotic. We also conducted a retrospective cohort study to assess clinician-centered outcomes.
Study population. We included children between 6 months and 12 years of age who had been diagnosed with AOM, GAS pharyngitis, or acute sinusitis using ICD diagnosis codes (Appendix Table 1) and prescribed an antibiotic. For GAS pharyngitis, the child also had a positive rapid streptococcal test. Children were excluded if they were diagnosed with multiple ARTIs during the index visit or another non-ARTI bacterial infection, or had been prescribed antibiotics by a provider in the network in the past 30 days.
Sampling strategy. A list of children meeting these criteria, as identified electronically in the EHR, was generated 2 times per week, and a sample was randomly selected stratified by ARTI diagnosis, antibacterial spectrum of the prescribed antibiotic, and type of primary care practice visit (an urban teaching practice compared with all other types of practices). Selection probabilities changed over the study period. The sample was stratified so that the cohort would have adequate numbers of children in each diagnosis and antibiotic exposure category. Children were enrolled between January 2015 and April 2016.
Data collection. The legal guardians of sampled children were contacted by telephone between 5 and 10 days after diagnosis and asked to participate in 2 telephone interviews. Interviews were conducted by 5 research assistants who were trained in obtaining informed consent, administering the structured telephone interview, and entering data. To contact as many sampled children as possible, the research assistants attempted contact at different times of the day over the course of the 6-day contact period.
At the time of telephone contact with the confirmed legal guardian, the eligibility criteria (specifically, the ARTI diagnosis, antibiotic prescription, and no antibiotic use in the past 30 days) were confirmed. Additionally, it was confirmed that the child had started the antibiotic treatment. If the child had not started the antibiotic course, the child was not enrolled. The legal guardian provided verbal informed consent to participate in the telephone interviews and separately provided consent for review of the child’s EHR data. After completion of the interview at initial contact between 5 and 10 days after diagnosis, a second telephone interview was conducted between 14 and 20 days after diagnosis. A date and time for the second interview was scheduled during the first interview. If the second interview was not successfully conducted for the scheduled time, then the research assistant continued contact attempts during the 7-day interview period.
After enrollment and data collection, it was decided to exclude children with GAS pharyngitis who were less than 3 years old at diagnosis, to reflect the AAP guidelines that recommend against antibiotic prescribing for GAS pharyngitis in this age group.55
Exposure. The exposure was the spectrum of antibiotic prescribed at the index visit, referred to as narrow-spectrum (exposed) or broad-spectrum (unexposed) antibiotics (Appendix Table 2). At the time of the telephone interview, the name of the antibiotic was confirmed and only children who had started the prescribed antibiotic were included.
Patient-centered outcomes. Patient-centered outcomes were obtained from legal guardians during the 2 telephone interviews. The primary outcome was the health-related QOL score obtained using the Pediatric Quality of Life Inventory (PedsQL) (Mapi Research Trust, Lyon, France; www.pedsql.org) Parent-Proxy Report Generic Core Scales 56-58 and Parent Report Infant Scales59 administered during the day 5 to 10 interview. Briefly, the PedsQL is a 23-item questionnaire assessing developmentally appropriate metrics related to core dimensions of health and role functioning. The questions are specific to the subject’s age: 1 to 12 months, 12 to 24 months, 13 to 24 months, 2 to 4 years, 5 to 7 years, and 8 to 12 years. Our primary outcome was the Total Scale Score, which is a summary score of physical, emotional, social, and academic functioning. The score range is zero to 100 and higher scores indicate a better health-related quality of life.
Secondary outcomes assessed were the presence of symptoms beyond 72 hours of treatment (third day postdiagnosis) and the occurrence of side effects including diarrhea, rash, upset stomach, or vomiting. For the persistence of symptoms, parents were asked about the presence of 3 or 4 specific symptoms relevant to the child’s diagnosis (AOM: fever, ear pain, decreased appetite; acute sinusitis: fever, face or head pain, runny nose, decreased appetite; GAS pharyngitis: throat pain, fever, decreased appetite) at the time of diagnosis. If the parent who reported the symptom had been present, the parent was asked if and when it had resolved. To assess the persistence of symptoms, we looked only at resolution of symptoms that would be reasonably expected to resolve with 72 hours of treatment so we did not include runny nose for acute sinusitis. Because children reported side effects only in the day 14 to 20 interview, the outcome was missing for children whose families could not be reached for that interview.
Additionally, among children who attended school or daycare, we assessed missed school or daycare and the need for additional child care (primary or secondary caretaker had to miss work or daily obligation and/or caretaker had to arrange for additional child care).
Covariates. Potentially important covariates were obtained from both the interviews and electronically abstracted from the EHR. Patient and visit characteristics obtained from the EHR included age (continuous by whole number year), sex, use of public insurance at the acute ARTI visit, season (winter: December-February; spring: March-May; summer: June-August; fall: September-November), prescription of antibiotic otic medication (for AOM only), visit occurred at urban/teaching practice, provider, and type of provider (physician with <10 years postresidency, physician with ≥10 years postresidency, nurse). The number of years postresidency was obtained from 2 sources: a file maintained by the primary care network director and by searching for the physician online.
The primary source of data for race and ethnicity was the day 14 to 20 interview. When families were not available for that follow-up interview, the data were obtained from the EHR. The race and ethnicity documented in the EHR was a combination of parent report and race as perceived by the practice staff. For analysis, we combined the race and ethnicity categories into a single, 4-level variable: Latino, White, Black, other, or mixed.
Lastly, during the day 5 to 10 interview, research assistants asked the parent which symptoms relevant to the specific diagnosis were present at diagnosis (number of symptoms: 0, 1, ≥2) and if the child attended school or daycare.
Analytic approach. We identified 2 important analytic issues and developed an approach, a priori, to address the challenges of potential confounding. First, previous research in this primary care network showed that antibiotic choice varied greatly by provider and that the variability could not be fully explained by differences in patient characteristics.12 Further, patient-reported outcomes may vary by provider through myriad mechanisms, such as the provider’s quality of care and office setting. Therefore, it was necessary to account for potential confounding by provider. Second, in addition to provider preference as a potential source of confounding, antibiotic spectrum may also be a product of patient characteristics. For example, in this primary care network, we previously documented variability in antibiotic prescribing by race.60 The patient-reported outcomes are also likely to be associated with these same patient characteristics. Therefore, it was necessary to account for potential confounding by patient characteristics.
We used 2 complementary analytic approaches to address these issues: (1) stratification by provider to address primarily potential confounding by provider, and (2) propensity score–based full matching to balance on patient characteristics and therefore to minimize patient-level confounding factors. 61,62 Using 2 distinct approaches that addressed different types of confounding allowed us to examine the robustness of model selection to both provider- and patient-level confounding. Both approaches were stratified by diagnosis, as we wished to contrast the effects of antibiotics across patients within each of the specified diagnoses.
Stratification by provider. In this approach, we stratified the cohort by diagnosis (AOM, GAS pharyngitis, acute sinusitis) and individual clinician to adjust for potential confounding by provider. Further, because primary care practices usually attract patients within a certain geographic area, by stratifying on provider this analysis also adjusted for some geographical and neighborhood factors. Considering the size of the network and the heterogeneity of patient characteristics across the 31 practice sites, this approach to confounding control resulted in many strata of varying size, some of which were small. Because we were unable to adjust for observed patient-level characteristics in samples from small strata, this approach does not address directly confounding by patient characteristics or severity of illness (confounding by indication). It does, however, consider patient characteristics indirectly owing to the relative homogeneity of patient characteristics within site from the neighborhood catchment of those sites. In this approach, for our primary, continuous outcome (the health-related QOL score obtained using the PedsQL), we used fixed-effects linear regression where strata were included as fixed effects to obtain the health-related QOL score difference between the exposure groups. The response model for the primary outcome included indicators for the age categories of the PedsQL questionnaires in order to account for differences in calibration of PedsQL scoring across the questionnaires. For binary outcomes, we used conditional logistic regression to estimate odds ratios and fixed-effects linear regression to estimate risk differences.63 Conditional logistic regression dropped patients if a stratum did not contain patients with differing outcomes; the reported sample size (n) reflects this loss of data. To estimate variability within the model, we implemented robust variance methods to estimate standard errors for each coefficient.
Propensity score–based full matching. To adjust for potential confounding by patient- and physician-level factors, we implemented propensity score–based full matching. The advantage of this approach is that it minimizes overall “distance” (differences) across patients by creating matched sets of different sizes and different ratios of treatment groups (ie, broad- versus narrow-spectrum antibiotics). 64 Unlike other matching approaches, full matching uses all subjects. As such, full matching functions in a manner similar to subclassification but with far greater ability to balance patient covariates by reason of the number and size of the resulting matched sets. The minimization of differences between patients occurs without regard to the outcome, and, as such, full matching emulates a process of heavily stratified randomization. However, this approach does not adjust for potential confounding by provider as well as the previous method does. Confounding by provider, therefore, is sacrificed here in favor of better adjustment for confounding by indication. Full matching of patient characteristics stratified by provider would address both sources of confounding (physician and patient characteristics), but that approach would have required large samples of patients within each physician’s practice to achieve good matching.
The full matching approach was a 2-step method: (1) propensity score models and full matching and (2) weighted response models. First, a propensity score model was estimated for each of the 3 diagnoses, using logistic regression with the dichotomous exposure (antibiotic spectrum) as the dependent variable. The regression included the covariates described above. Propensity-score models did not include children who were missing the patient-reported outcome; therefore, matching was repeated for each outcome. Children missing race/ethnicity data were excluded from this approach. Using the propensity score, full matching (using R package Optmatch 65) optimally matched children who received narrow-spectrum antibiotics and broad-spectrum antibiotics in varying numbers within each matched set. We limited the broad-to-narrow ratio within matched sets from 5:1 to 1:5 to improve precision of final estimates.61 Within the matched sets, weights for the average treatment effect were calculated. Balance of matching was assessed by comparing weighted means for each covariate between exposure groups stratified by diagnosis. If imbalance remained, these factors and interactions of these factors with diagnosis were included in the response models. In the second step, the weighted response models were fit. For our primary outcome, which was continuous, we used weighted linear regression to obtain the health-related QOL score difference between the exposure groups. The linear regression included indicators for the age categories of the PedsQL questionnaires in order to account for differences in calibration of PedsQL scoring across questionnaires, which differed by patient age category. For binary outcomes, we used weighted logistic regression to estimate odds ratios and weighted logistic with marginal standardization to estimate risk differences. Variances were parametric-based estimates that account for the matching.66 Variances of risk differences were based on the delta method transformation of variances from the logistic regression.67
Sensitivity analyses. Two sensitivity analyses were conducted. The first was designed to assess the potential impact of confounding by unobserved factors on the results observed for the primary outcome: the PedsQL score. We implemented a formal sensitivity analysis as described by Ding and VanderWeele. 68 Using the risk difference point estimate of 1.4, the estimated superiority of narrow-spectrum over broad-spectrum antibiotics on the PedsQL in the matched analysis, we investigated the degree of confounding necessary to shift this estimate so that broad-spectrum antibiotics would be superior by 1.0.
The second sensitivity analysis was designed to account for missing responses for occurrence of side effects due to loss to follow-up. Missingness was accounted for using inverse probability weighting, where weights were generated from a logistic regression model in which the dependent variable was whether the side effects outcome was observed. These weights were combined with the propensity score weights generated as described previously—but for all patients in this analysis, as opposed to just those with the observed outcome. The final weight was used in the response models described previously to estimate the parameter of interest under a missing at random assumption.
Study population. Using a more standard, clinician-centered approach, we then compared the effectiveness of broad- versus narrow-spectrum antibiotics for these common bacterial ARTIs. The goal of this study was to provide a complementary approach to answering this question and to offer a comparison of results between patient- and clinician-centered outcomes. To this end, we conducted a retrospective cohort study of a full calendar year of data including all patients with the targeted conditions across the entire primary care network. Although the inclusion criteria and antibiotic exposure definitions were the same, the outcomes for this clinician-centered cohort were treatment failure identifiable in the EHR, as defined below.
We included children between 6 months and 12 years old who had been diagnosed with a new ARTI at an in-person visit to a primary care practice within the network in 2015. The diagnosis was identified using ICD diagnosis codes, versions 9 or 10 (Appendix Table 1). For a GAS pharyngitis diagnosis, a positive rapid streptococcal test was required. The visit where the diagnosis occurred will be called the “index visit.”
We excluded children (1) whose guardian had declined consent for review of the child’s EHR during the telephone interview to assess patient-centered outcomes described previously, (2) who were diagnosed with more than 1 of the ARTI diagnoses of interest, (3) who were not prescribed an appropriate antibiotic (Appendix Table 2) for an appropriate duration to treat the diagnosis (5-15 days except for Azithromycin, which was 3-7 days), (4) who were prescribed multiple antibiotics, (5) who were diagnosed with other bacterial infections at the same visit, (6) who had been prescribed antibiotics in the previous 30 days, (7) who had received an ARTI diagnosis in the previous 30 days, or (8) who had previously been diagnosed with a complex chronic condition (using a published ICD9 code-based algorithm69). Among children diagnosed with GAS pharyngitis, children less than 3 years old were also excluded. Data from the index visit and the 30 days after the index visit were electronically extracted from a comprehensive EHR used exclusively by all practices for all office and telephone encounters.
Exposure. The exposure was the spectrum of antibiotic prescribed at the index visit: narrow-spectrum (exposed) or broad-spectrum (unexposed; Appendix Table 2).
Outcomes. There were 2 primary outcomes: treatment failure up to 30 days after diagnosis and the occurrence of side effects up to 30 days after diagnosis.
The definition of treatment failure varied, depending on when the event occurred during the 30-day period: (1) during the “effective” duration of the index antibiotic course and (2) after the “effective” duration of the index antibiotic course. The “effective” duration equaled the prescribed duration, except for Azithromycin where the “effective” duration was twice the length of the prescribed duration (to account for the pharmacokinetics of this drug). For children with missing data about the prescribed duration, 10 days was imputed for all antibiotics except for Azithromycin, where 5 days (10 days of “effective” therapy) was imputed. The imputation was necessary to define appropriate time windows for the assessment of treatment failure. During the “effective” therapy period, treatment failure was defined as persistence of symptoms requiring a new antibiotic: an in-person or telephone contact with the same ARTI diagnosis with a switch in antibiotic therapy where the progress notes noted persistence of symptoms or concern for failure of original antibiotic. Progress notes from outpatient and emergency department encounters defined as potential treatment failures were manually reviewed by 1 of 2 reviewers, both of which were registered nurses with more than 10 years of experience in hospital epidemiology. After the “effective” therapy period, treatment failure was defined as reoccurrence: an in-person or telephone contact with the same ARTI diagnosis with receipt of any systemic, ARTI-targeted antibiotic (even if the same as the initial therapy). Lastly, all hospitalizations (inpatient and observation) with an antibiotic during the 30-day period were reviewed by author JSG, a pediatric infectious diseases specialist, for the occurrence of a complication requiring hospitalization resulting from treatment failure (eg, peritonsillar abscess, mastoiditis, brain abscess) to be described as a secondary outcome. Treatment failures without complication were also noted during the review of hospitalizations and included in the primary outcome. Treatment failure was not assessed until 2 days after diagnosis because we rationalized that antibiotic therapy for less than 2 days with the original prescribed antibiotic was not sufficient time to truly assess effectiveness of therapy.15,55,70
The definition of the occurrence of side effects also varied depending on the timing during the 30-day period because different side effects typically occur within different time windows. During the “effective” course (ie, the 10-day exposure window), side effects assessed were diarrhea, candidiasis, non-Candida rash, other/unspecified allergic reaction, other/unspecified adverse effect, and vomiting. Candidiasis was also assessed for 1 week after the completion of the “effective” course. Diarrhea was assessed for the entire 30-day period. Side effects were identified by 3 methods: (1) manual review of progress notes from encounters with the same ARTI diagnosis and a new antibiotic during the “effective” course (same outpatient and emergency department encounter review described above for persistence of symptoms), (2) Intelligent Medical Object codes added to the EHR during appropriate period at any type of encounter (Appendix Table 3), and 3) new antibiotic allergy noted in EHR.
Covariates. Patient demographic variables included age (continuous in years), sex, race and ethnicity (Latino, White, Black, other, or mixed race), and use of public insurance (if insurance was missing, insurance was obtained from most recent visit before index visit). Index visit variables included were season (winter: December-February, spring: March-May, summer: June-August, fall: September-November), prescription of antibiotic otic medication (for AOM only), visit occurred at urban/teaching practice, provider, and type of provider (physician with <10 years postresidency, physician with ≥10 years postresidency, nurse).
Analytic approach. The analytic issues identified and discussed above in the analytic approach for the patient-centered outcomes also applied to the approach for the clinician-centered outcomes. Briefly, we identified confounding by provider and confounding by patient characteristics as important analytic issues. Therefore, we used 2 complementary approaches: (1) stratification by provider to address primarily potential confounding by provider, and (2) propensity score–based full matching to balance on patient characteristics and therefore to minimize patient-level confounding factors. 61,62 Both approaches were stratified by diagnosis, as we wished to contrast the effects of antibiotics across patients within each of the specified diagnoses.
Stratification by provider. We stratified the cohort by diagnosis and provider to adjust for potential confounding by provider. We used conditional logistic regression to estimate odds ratios and fixed-effects linear regression to estimate risk differences between broad- and narrow-spectrum antibiotics.63 Conditional logistic regression dropped patients if a strata did not contain patients with differing outcomes; therefore, the reported sample size (n) reflects this loss of data. To estimate variability within the model, we implemented robust variance methods to estimate standard errors for each coefficient.
Propensity score–based full matching. To adjust for potential confounding by patient- and physician-level factors, we implemented propensity score–based full matching. A propensity-score model was estimated for each of the 3 diagnoses, using logistic regression with the dichotomous exposure (antibiotic spectrum) as the dependent variable. The regression included the covariates described above. Children missing race/ethnicity data were excluded from this approach. Using the propensity score, full matching (using R package Optmatch65) optimally matched children who received narrow-spectrum antibiotics and broad-spectrum antibiotics in varying numbers within each matched set. Because the AOM cohort was too big to implement matching without subsetting, we conducted full matching with exact matching on sex and public insurance. For GAS pharyngitis, practice type remained imbalanced after fitting the propensity score model and completing full matching. In a second iteration, we exactly matched on practice type before conducting full matching. We limited the ratio of matched sets variability by diagnosis: AOM 1:9, acute sinusitis 1:5, and GAS pharyngitis 1:13. Within the matched sets, weights for the average treatment effect were calculated. Balance of matching was assessed by comparing weighted means for each covariate between exposure groups stratified by diagnosis. In the second step, we used weighted logistic regression to estimate odds ratios and weighted logistic with marginal standardization to estimate risk differences. Variances were parametric-based estimates that account for the matching.66 Variances of risk differences were based on the delta method transformation of variances from the logistic regression.67
Analyses were performed using Stata version 14.2 (Stata Corp) and R version 3.1.1.
A total of 109 parents completed in-person interviews (Table 3). Parents had a median age of 38 (interquartile range [IQR], 30-46 years). Nearly half self-identified as White and the majority of all patients as non-Hispanic or Latino. Approximately a third of parents had a college degree or higher and slightly more than half were employed. Close to half the parents had a child aged 5 or younger, and most (85%) had previously received antibiotics for a child. A total of 24 children completed an interview (13 females and 11 males). Four children were recruited from the YAC, 10 from the urban practice, and 10 from the suburban practice. Children had a median age of 14 years (IQR, 12-15 years).
Characteristics of 109 Parent Interview Respondents.
In general, the parents we interviewed recognized the efficacy of antibiotics and expressed gratitude to have them “when they are needed” (Table 4). At the same time, our respondents described a general feeling of wariness when their child is prescribed an antibiotic, the sense that antibiotics are a “necessary evil,” and a strong desire to be as judicious as possible regarding their use. As one parent explained:
Themes Identified in the Data With Illustrative Verbatim Comments From Parents.
I have a thing about using [antibiotics]. I don’t like to medicate unless I have to. I do think about not overmedicating him with antibiotics. Like, right now, if the doctor says he is going to need antibiotics, I am going to think, “Ugh, we just had them a month ago.” He had an infection in a surgical site. But there is not much you can do about it if he has to have one.
The majority of parents in our study believed that antibiotic overuse is a problem and had heard about antibiotic resistance or “superbugs” (n = 90). Parents understood the general concept of antibiotics no longer working to treat common infections, with variation in the level of sophistication used to describe resistance. Most parents perceived that the body becomes “immune” to antibiotics, only a few (n = 7) parents mentioned resistant organisms. Few parents expressed concerns about antibiotic resistance directly affecting them or their child. Multiple parents said that they would become concerned about resistance only if their child received antibiotics frequently.
When asked about what goes through their mind when their child is prescribed an antibiotic, many parents reported that they had a very strong preference for “natural” remedies for ARTI symptoms, including changing the child’s diet, using cold compresses for pain, steam to loosen secretions, and essential oils. For some parents, this preference was driven by a concern that their child “would become overmedicated somehow.” This concern was heightened for parents of children less than 2 years of age, who were worried about “putting too much medicine” into the body of a young child. We observed that for the majority of parents expressing a preference for “natural” remedies, this desire was embedded in a perception that any prescription medication was risky. As one parent said:
I worry about antibiotics. But I feel like sometimes a lot of the medicines we are given are not called for. Like, I think it does more damage sometimes than trying to fix problems. I just think everything we take is going to harm us in some way.
We observed that these perceptions emerged across parents of different sociodemographic characteristics, including race, ethnicity, level of education, and location of practice.
We identified 4 outcomes related to antibiotic use that were repeated across the majority of parent respondents to inform the comparative effectiveness study: child sleep quality, missed school and work, child suffering, and speed of symptom resolution. Multiple parents described how 1 of the most burdensome aspects of having a sick child was the impact of illness on the child’s sleep quality. When we asked about expectations for the visit with the pediatrician, many parents expressed a desire for a solution to help improve the child’s disrupted sleep due to congestion, coughing, or pain. Parents described being concerned about their child having to miss school due to illness and the burden on them or their spouse of having to miss work to take care of the sick child. Most parents said that one of the hardest parts of having a sick child was witnessing them suffer, be in pain, or be unhappy. Finally, many parents expressed that the speed of symptom resolution was important.
A summary of responses to our closed-ended questions assessing the level of concern that parents have about antibiotic treatment failure and side effects appears in Table 5. We found that parents were not concerned about antibiotic treatment failure; somewhat concerned about their child developing diarrhea, an upset stomach, or having an allergic reaction; and slightly concerned about their child getting a rash.
Answers to Closed-Ended Questions Asked of Parents.
Overwhelmingly, the children in our sample did not know what an antibiotic was, did not know if they had ever taken one before, or did not have particularly strong opinions about antibiotic therapy, with the exception of preferences on the flavor of liquid antibiotics. When we asked children how ARTI symptoms impacted their daily life they told us that they worried about missing school, being uncomfortable, being unhappy, and not sleeping well, and that they wanted to get better as soon as possible so they could return to their normal routine. A summary of child responses to closed-ended questions about the importance of various outcomes related to antibiotic therapy is summarized in Table 6. The children in our sample identified speed of symptom resolution, the ability to return to their normal routine quickly, missed school, and sleep quality as very important outcomes. They also expressed concern about their parents having to miss work.
Answers to Closed-Ended Questions Asked of Children.
Based on these findings we determined that we should include measures in our comparative effectiveness study that captured disrupted sleep and/or fatigue, child suffering, missed school and work, and speed of symptom resolution. To determine which measures to use, we convened a meeting of the research team, including our patient stakeholder, to discuss how to best capture these outcomes. We reviewed these findings and the literature to determine whether we should use already validated measures or create our own. For missed school and work and speed of symptom resolution we created questions that captured data on whether children and parents missed school or work and the duration of symptoms. For sleep and child suffering, we determined that the PedsQL was the most appropriate measure to capture the impact of an ARTI on health-related QOL. This measure captures social, physical, emotional, and academic functioning. It has been demonstrated to be feasible and reliable, and to have good construct and discriminant validity and responsiveness in measuring short-term outcomes after minor acute episodes of illness or injury.71 We considered validated measures of sleep disruption but determined, in close collaboration with our family partners, that these measures were too long and complex and would likely be burdensome to families if administered during the telephone interview.
Based on data electronically extracted from the EHR during the study period, we identified 58 488 children who fit inclusion and exclusion criteria, and 10 296 of them were randomly selected in the biweekly stratified samples Figure 1. We successfully contacted, confirmed eligibility, obtained consent, and completed the day 5 to 10 interview with the primary care taker for 2472 children, including 1100 with AOM, 667 with acute sinusitis, and 705 with GAS pharyngitis. The enrolled children had characteristics similar to those of the other children sampled who were not enrolled (Appendix Table 4).
Flowchart of the Patient-Centered Outcomes Cohort. Abbreviation: EMR, electronic medical record.
Overall, 65% (1604) of children in the final cohort were prescribed narrow-spectrum antibiotics (AOM 62%, acute sinusitis 63%, and GAS pharyngitis 71%). The day 14 to 20 interview was completed with 2096 (85%) of the 2472 children. The cohort of children that missed the day 14 to 20 interview had a higher proportion of White children and a lower proportion of children with public insurance than the cohort that completed both interviews (Appendix Table 5).
Tables 7A, 7B, and 7C describe the demographic and clinical characteristics for the children included in the patient-centered outcomes cohort stratified by diagnosis. For AOM and GAS pharyngitis, differences by treatment group were noted for race/ethnicity and public insurance; however, these differences reflect the population that seeks care at urban/teaching practices, where providers have been shown to have a strong preference for narrow-spectrum antibiotics based on our previous work in this network. Across all 3 diagnoses, the differences in antibiotic treatment choice by season were a product of changes in the sampling probabilities over the study period.
Patient and Clinical Characteristics of Children Diagnosed With Acute Otitis Media in the Patient-Centered Outcomes Cohort.
Patient and Clinical Characteristics of Children Diagnosed With Acute Sinusitis in the Patient-Centered Outcomes Cohort.
Patient and Clinical Characteristics of Children Diagnosed With Group A Streptococcal Pharyngitis in the Patient-Centered Outcomes Cohort.
As outlined above, we used 2 complementary approaches to address potential confounding by provider and patient. Utilizing 2 distinct approaches that addressed different types of confounding allowed us to examine the robustness of model selection to both provider- and patient-level confounding. In the stratified approach, the cohort was stratified by diagnosis and provider. There were 266 providers (median number of children per provider 7, range 1-47) resulting in 595 total strata (AOM, 221; acute sinusitis, 151; and GAS pharyngitis, 223).
In the propensity score–based full matching approach, 41 children missing race/ethnicity were excluded. Overall, 2431 children were included in the full matched approach. After matching, imbalance remained for some covariates for GAS pharyngitis. Specifically, practice type (urban/teaching) and race/ethnicity were imbalanced after matching for each outcome, and public insurance was imbalanced after matching for the side effects outcome (see Appendix Tables 6-10). We reasoned that the remaining imbalance of race and insurance was driven by practice type (urban/teaching practice versus nonurban/teaching); therefore, practice type, diagnosis, and an interaction term for practice type by diagnosis was included in response models for the propensity score–based full matched approach.
The PedsQL served as our primary, patient-centered outcome and was administered during the first interview. One subject did not answer enough questions on the PedsQL to obtain a score. For the remaining 3022 children, the average score was 90.2 (SD, 10.6) (Appendix Figure 1). The average score was 91.6 (SD, 9.3) for children who received narrow-spectrum antibiotics and was 90.2 (SD, 10.5) for children who received broad-spectrum antibiotics. Table 8 displays the results for the PedsQL score from both analytic approaches. In both approaches, narrow-spectrum antibiotic exposure was associated with a statistically significantly higher score; however, it is unclear if this small change truly indicates a higher health-related QOL for children prescribed narrow-spectrum antibiotics as compared with those who were prescribed broad-spectrum antibiotics.
Results for Primary Patient-Centered Outcome, PedsQL Health-Related QOL Total Scorea .
Using the point estimate of 1.4-point score difference from the matched analysis (narrow spectrum superior to broad spectrum), we investigated the degree of confounding that would be necessary to shift this estimate so that broad-spectrum antibiotics would be superior by 1.0 points. The minimum strength of association of confounder and outcome and confounder and choice of antibiotics would correspond to a relative risk of 1.8 for both dimensions of confounder association. Because our analysis was matched for observed patient characteristics, this degree of confounding would have to persist after these adjustments.
Secondary patient-centered outcomes. Table 9 shows the results for the secondary outcomes from both analytic approaches. Narrow-spectrum antibiotics were not worse than broad-spectrum antibiotics across all secondary outcomes. The results are consistent across both analytic approaches (stratified and full-matched). For the occurrence of side effects, 11.5% (240/2094) reported rash, 20.0% (419/2093) reported diarrhea, and 6.2% (130/2090) reported vomiting or upset stomach. Overall, 28.8% (603/2093) of children had a side effect reported in the interview. Receiving narrow-spectrum antibiotics was associated with a significantly reduced risk of side effects compared with broad-spectrum antibiotics (full matched analysis: RD [risk difference] = –11.4%; 95% CI, –16.4 to –6.4). The results from the sensitivity analysis that accounted for the missingness of this outcome were similar (RD = –12.9; 95% CI, –18.0 to –7.7). The need for additional child care, required by 32% of families, was the only outcome where the point estimate favored broad antibiotics (stratified analysis: RD = 0.2%; 95% CI, –5.3 to 5.6; odds ratio [OR], 1.01; 95% CI, 0.77-1.31). However, these results were not statistically significant and the confidence bounds suggest that almost as much evidence favors narrow as favors broad for this outcome.
Results for Secondary Patient-Centered Outcomesa .
Overall results. Of 712 418 in-person encounters in the primary care network in 2015, 57 004 children met inclusion criteria. After applying exclusion criteria, 30 159 children remained in the analytic cohort (19 179 AOM, 4234 acute sinusitis, and 6746 GAS pharyngitis; Figure 2). Overall, 85% (25 852) of children in the cohort were prescribed narrow-spectrum antibiotics (AOM, 86%; acute sinusitis, 80%; and GAS pharyngitis, 88%).
Tables 10A, 10B, and 10C describe the demographic and clinical characteristics for the children included in the clinician-centered outcomes cohort, stratified by diagnosis. Consistent with the patient-centered outcomes cohort, differences by treatment group were noted for race/ethnicity and public insurance for AOM and GAS pharyngitis.
Patient and Clinical Characteristics of Children Diagnosed With Acute Narrow-spectrum Otitis Media in the Clinician-Centered Outcomes Cohort.
Patient and Clinical Characteristics of Children Diagnosed with Acute Sinusitis in the Clinician-Centered Outcomes Cohort.
Patient and Clinical Characteristics of Children Diagnosed with Acute Sinusitis in the Clinician-Centered Outcomes Cohort.
In the stratified approach, the cohort was stratified by diagnosis and provider. There were 336 providers (median number of children per provider, 54; range, 1-421), resulting in 798 total strata (304 AOM, 202 acute sinusitis, and 292 GAS pharyngitis).
In the propensity score–based full matched approach, 73 children with missing race/ethnicity were excluded, resulting in 30 086 children included in the full matched approach. After matching, all covariates, stratified by diagnosis, were successfully balanced (Appendix Table 11).
Overall, 2459 (8.2%) children experienced treatment failure, and 1039 (3.5%) children experienced at least 1 side effect. Table 11 shows the results for both analytic approaches. In both approaches, there was no important difference in treatment failure between treatment groups. For side effects, narrow-spectrum antibiotics were associated with a reduced risk in both analytic approaches, compared with broad-spectrum antibiotics.
Results for Clinician-Centered Outcomesa .
Results by diagnosis. Table 12 displays the results stratified by diagnosis. The results stratified by diagnosis for occurrence of side effects are comparable to the pooled results of all diagnoses; however, the estimated effect of antibiotic spectrum on treatment failure differed by diagnosis. Specifically, for AOM, narrow-spectrum antibiotics were associated with a reduced risk of treatment failure compared with broad-spectrum antibiotics (matched analysis RD = –1.5%; 95% CI, –2.9% to –0.1%). Conversely, for GAS pharyngitis, the point estimate suggested an increased risk of treatment failure associated with narrow-spectrum antibiotics, though it was not statistically significant at conventional levels (matched analysis RD = 1.7%; 95% CI, 0.0%-3.5%). As described in the methods, the definition of treatment failure included persistence of symptoms (a new antibiotic prescription for unresolved illness before the completion of the initial antibiotic course), reoccurrence of infection (indicated by the same diagnosis coupled with an antibiotic prescription following completion of the initial antibiotic course until 30 days after diagnosis), or hospitalization. For all 3 conditions, the majority of treatment failure episodes were reoccurrence (AOM, 84%; acute sinusitis, 88%; and GAS pharyngitis, 95%). Hospitalizations were rare.
Results for Clinician-Centered Outcomes Stratified by Diagnosisa .
We identified only 2 children who had a complication leading to hospitalization, both of whom were originally diagnosed with GAS pharyngitis. One child was prescribed cefdinir at the time of diagnosis but was hospitalized 2 days later with a retropharyngeal abscess and was given intravenous clindamycin. The second child was prescribed amoxicillin at the time of diagnosis and was hospitalized 12 days later, after the GAS pharyngitis had resolved, with post-streptococcal pharyngitis reactive arthritis.
Using a large, diverse, pediatric primary care network, this group of studies (1) identified and prioritized relevant patient-centered outcomes and (2) applied these outcomes to compare the effectiveness of the 2 prevailing treatment strategies for the most common childhood infections. Overall, broad-spectrum antibiotics did not perform better than narrow-spectrum antibiotics on any of the identified patient-centered outcomes and, on our primary measure of quality of life, narrow-spectrum antibiotics performed better than broad-spectrum antibiotics. These data support the use of narrow-spectrum antibiotics for the treatment of ARTIs in children and reveal that many children unnecessarily receive antibiotics that are more likely to cause direct patient harm and promote antibiotic resistance.
The rationale for the choice of targeted conditions and comparators was multifactorial. First, these conditions are highly prevalent. Antibiotics are overwhelmingly the most common prescription medications given to children, and the 3 targeted conditions—ear, sinus, and throat infections—account for roughly two-thirds of all antibiotic prescribing in ambulatory pediatrics. Notably, prior studies have demonstrated that these drugs are substantially overused.47 For example, in 2012, children less than 2 years of age in the United States received more than 1300 antibiotic prescriptions per 1000 population, while Swedish children of the same age received fewer than 500 prescriptions/1000.72,73 Second, there is a clear lack of consensus among prescribers. Despite AAP guidelines specifying narrow-spectrum agents such as penicillin or amoxicillin for most children with presumed bacterial ARTIs, practice patterns have not followed these guidelines.9 This suggests that clinicians and/or patients might think that broad-spectrum antibiotics offer a therapeutic advantage. Third, the public health impact of this phenomenon is profound. Antibiotic overuse leads to the emergence of antibiotic-resistant pathogens and Clostridium difficile infection, both of which have been demonstrated to occur as a direct result of community-based antibiotic prescribing. 74 Furthermore, inappropriate antibiotic use can lead to adverse drug effects such as diarrhea, rash, drug–drug interactions, and allergic reactions that are at least a nuisance and at worst life threatening.24 In addition to these harmful but potentially transient side effects, systemic antibiotic exposure has been shown to cause changes in the composition and function of the gut microbiome.75 The association between this “dysbiosis” and a growing number of chronic health conditions demands investigation of the potential impact of inappropriate antibiotic choice on long-term health.
Deriving and utilizing patient-centered outcomes most relevant to children with ARTIs was fundamental to this work. To this end, a combination of open-ended (to identify themes without influencing the reporters) and closed-ended interviews (to help conform and prioritize outcomes deemed potentially important a priori) were conducted, targeting parents of children presenting to primary care clinics with signs and symptoms of ARTIs and otherwise healthy children, which was intended to maximize both the relevance and richness of these data. From open-ended interviews, parents and children identified disrupted sleep, missed school/work, child suffering, and speed of symptom resolution as important outcomes related to therapy for ARTIs. From closed-ended interviews, parents confirmed the importance of diarrhea, gastrointestinal distress, and fear of allergic reaction—outcomes we had previously identified as important. These data led us to use the PedsQL, a validated instrument designed to measure health-related QOL in children, for the primary outcome coupled with a combination of specific secondary outcomes to capture the themes derived from parent/patient interviews.
Comparative effectiveness studies of different therapeutic approaches that rely exclusively on outcomes observed by treating clinicians risk missing differential rates of symptom resolution and adverse drug effects that are meaningful to patients and their families but are not recorded in the medical record. This could occur either because the events did not lead to a medical encounter (eg, caused diarrhea that is manageable at home) or because the clinician did not recognize their impact of patient/family life (eg, sleep disruption, missed work/school) or may not have documented it in the medical record. The complementary retrospective cohort study, performed using exclusively EHR data as part of this same project, provided evidence of these limitations. For example, although both cohorts revealed increased rates of adverse drug effects in children exposed to broad-spectrum antibiotics, the incidence of adverse drug effects was tenfold higher when using the more granular, patient-centered approach than when relying solely on the medical record. These results demand a more patient-centered approach to collecting outcome data, particularly in the age of the EHR, in which these data elements could be collected in a structured and uniform way that would better inform clinician decision making.
We conducted a complementary analysis to assess how our patient-centered results compared with more standard clinical outcomes. In addition to the difference in apparent prevalence of adverse drug effects, the complementary “clinician-centered”/EHR analysis had 1 inconsistency in the direction of the treatment effect in a subanalysis of children with GAS pharyngitis, which favored broad-spectrum therapy. Although the difference was small (1.7% points in the full matched analysis) and not statistically significant, this result is notable. Given previous reports that broad-spectrum antibiotics, such as cephalosporins, decrease the rate of GAS carriage compared with first-line recommended agents (penicillin or amoxicillin), it is possible that this apparent advantage of broad-spectrum therapy is an artifact of repeat GAS testing in asymptomatic patients and not truly treatment failure. Again, because this more traditional comparative effectiveness study relied only on data routinely collected in the medical record (and, in this specific case, defined treatment failure as a repeat encounter with the same diagnostic code plus new antibiotic prescription), this approach could not distinguish between repeat infection and persistent but asymptomatic carriage, for which antibiotic therapy is not indicated. Thus, if children initially receiving narrow-spectrum therapy were more likely to be switched to a broad-spectrum drug in the face of retesting patients with persistent carriage (because those on broad-spectrum agents were already on broad-spectrum therapy), this finding could be an artifact of the medical-record-based outcome definition. The results from the prospective cohort, finding no difference between broad- and narrow-spectrum agents for GAS pharyngitis in the more granular and patient-centered outcomes, is consistent with this explanation. However, further study is required to reconcile this discrepancy.
This study had limitations. Because we relied on clinician diagnosis plus an antibiotic prescription to identify bacterial ARTIs, a substantial proportion of children likely had viral infections. Such misclassification could have blunted the ability to detect a difference between antibiotic choices but would have likely been nondifferential with respect to the direction of point estimates. However, this approach reflects the true prevalence of antibiotic prescribing and, therefore, captures its impact on symptom resolution, adverse effects, and quality of life. The successful phone contact rate for the prospective cohort study was only 30%. Although not surprising, considering that patients were not previously informed about the study before the initial enrollment calls, this low probability of contact could have led to bias in estimates if the chances of contact are related to the type of antibiotic prescribed. However, when we compared the full sample of eligible participants to the children included in the cohort, there were not notable differences in patient characteristics. To help mitigate this potential source of bias, analyses were stratified by practice and adjusted for a variety of patient clinical and demographic characteristics. Our primary patient-centered outcome was the PedsQL, but we were not able to obtain a baseline measure at the time of diagnosis so we could not assess the differential effect by antibiotic spectrum on the change in PedsQL. It is possible that baseline PedsQL varied by antibiotic spectrum due to comorbidities or severity of illness not captured by observed covariates. However, this is not likely, based on the results of our sensitivity analysis for an omitted confounder. To the extent that observed covariates reflect or are proxies for omitted confounders, a relative risk of 1.8 for residual association is relatively strong and suggests that our finding is robust to omitted patient-level confounders, such as severity of illness, that could mask superiority of broad-spectrum antibiotics. Although a validated instrument, the PedsQL was not designed to specially assess child suffering following ARTI. Further, although statistically significant, the small differences observed between patients treated with narrow- and broad-spectrum antibiotics are likely not clinically meaningful.
Fifteen percent of families who completed the day 5 to 10 interview did not complete the day 14 to 20 interview. However, only 1 outcome (side effects) and 2 covariates (race and ethnicity) were assessed at the second interview. A sensitivity analysis that accounted for missingness of the side effects account yielded similar results.
Although this study was conducted in a large primary care network including patients and clinicians from diverse geographic (urban, rural), socioeconomic, and racial groups, the results might not be generalizable outside of this patient population or region.
Not applicable.
As stated above, our results were largely consistent across all 3 conditions. There was, however, a small but statistically insignificant difference in the direction of the point estimate in the subgroup of children with streptococcal pharyngitis . Because this was observed only in the EHR study and not confirmed in the patient-centered analyses, we hypothesize that this represents an artifact of the increased likelihood of children with persistent streptococcal carriage being treated with broader-spectrum antibiotics after initial therapy with a narrow-spectrum agent, with the goal of eradication—a practice that is not recommended by the AAP.
Future work on this topic will include dissemination of these data throughout this network and beyond. We plan to apply for PCORI funding to formalize this process. Additionally, we plan to extend this approach to compare children who receive antibiotics to those who do not receive antibiotics for common ARTIs in an effort to further reduce unnecessary antibiotic use.
We identified that parents and children with ARTIs were most concerned with the effect of these infections on quality of life issues, including child suffering, missed school or work, and sleep disruption. Using these outcome metrics, generated in partnership with patients and their caregivers, broad-spectrum agents offered no benefit over narrow-spectrum agents for the treatment of the most common pediatric ARTIs, but they were associated with significantly more adverse drug effects. These data confirm and extend the recommendations to use narrow-spectrum antibiotics for most children with AOM, acute sinusitis, and streptococcal pharyngitis , a choice that will maximize patient outcomes while reducing unnecessary antimicrobial resistance pressure, adverse drug effects, and health care costs.
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#CE-1304-7279). Further information available at: https://www.pcori.org/research-results/2013/comparing-broad-and-narrow-spectrum-antibiotics-children-ear-sinus-and-throat
Gerber JS, Ross RK, Bryan M, et al. (2018). Comparing Broad- and Narrow-Spectrum Antibiotics for Children with Ear, Sinus, and Throat Infections. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/8.2018.CE.13047279
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
