An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Lisa I. Iezzoni, MD, MSc, Dennis Heaphy, MEd, MPH, and Karen Donelan, ScD.
Author Information and AffiliationsPersons who are dually eligible for Medicare and Medicaid generate disproportionately high costs. Individuals younger than 65 years of age who qualify for Medicare because of disability are especially costly, as they require expensive long-term services and supports (LTSS) needs. To explore efficient health care delivery models for these beneficiaries, the Centers for Medicare & Medicaid Services (CMS) initiated demonstration programs for dually eligible individuals. The Massachusetts One Care program is CMS's only demonstration program nationwide that targets younger beneficiaries: individuals aged 21 to 64 years. As CMS and Massachusetts Medicaid designed One Care, which involves dually capitated payments to plans and covers all services including LTSS, local disability advocates voiced concerns about threats to care quality. They argued that persons with disability should be the ones leading efforts to assess One Care quality, highlighting fears related to independent living in communities.
This study aimed to test the comparative effectiveness for improving patient-reported health care experiences of 2 informational interventions in a 12-month period: (1) the “YES Health: Your Experience, Speak up for better health care” initiative, in which disability advocates developed brief topical surveys and gathered information from One Care enrollees with significant physical disability or serious mental illness; and (2) the Persons With Disability Quality Survey (PDQ-S), developed collaboratively with persons with disability.
We focused on English- and Spanish-speaking One Care members with either serious mental illness or significant physical disability. This cluster randomized controlled trial randomly assigned 27 primary care practices with ≥50 One Care members meeting these specifications to 3 study arms differing by information provided to practice directors and primary care providers (PCPs): (1) quarterly YES Health reports plus results from baseline administration of PDQ-S to 720 enrollees before YES Health implementation; (2) PDQ-S results only; and (3) no study information. We administered PDQ-S again 1 year later and used difference-in-differences analyses of results across the 2 years to assess intervention outcomes. After YES Health concluded, we surveyed the 221 PCPs listed as serving sample members at the 27 practices.
Led by individuals with significant physical disability or serious mental illness, YES Health engaged 45 persons—with disabilities from the 9 randomly assigned practices—in surveys evaluating their One Care quality. Topics selected for the quarterly YES Health quality surveys were One Care plans and care teams, communication with clinicians, LTSS, and transportation. However, PCPs at the 9 study arm 1 practices did not engage with the disability advocates, despite repeated outreach. With minimal exceptions, we found no differences across the 3 study arms in PDQ-S member-reported outcomes. Of the 110 eligible PCPs who responded to the survey (54.4%), few reported knowing about YES Health or baseline PDQ-S results.
Individuals with disability successfully designed and implemented YES Health. Nonetheless, the inability to engage PCPs in collaborative interactions likely contributed to YES Health having no effect. Future research should consider approaches toward developing effective patient-physician partnerships to improve care.
This study may have limited generalizability beyond the context of dually eligible individuals aged <65 years in integrated care delivery systems with dually capitated reimbursement.
Persons who are dually eligible for Medicare and Medicaid compose 16% of Medicare beneficiaries and 15% of Medicaid recipients; however, this population generates 27% of Medicare spending and 39% of Medicaid expenditures.1 All dually eligible persons must meet Medicaid low-income requirements. Nearly two-thirds of dually eligible individuals are aged 65 or older, receiving Medicare through “old age” provisions. The other third generally qualifies for Medicare through entitlement for Social Security Disability Insurance (eg, disabled former workers).1 This younger population is particularly costly, with substantial needs for both medical care and long-term services and supports (LTSS) to facilitate independent living in the community and participation in daily life activities.
To control costs while maintaining care quality for dually eligible and other resource-intensive populations, the 2010 Patient Protection and Affordable Care Act (ACA) mandated experiments with innovative health care delivery models. To initiate and oversee these experiments, the ACA created the Center for Medicare and Medicaid Innovation (CMMI). In July 2011, in collaboration with another newly mandated office—the Medicare-Medicaid Coordination Office (MMCO)—CMMI announced its financial alignment demonstration for dually eligible beneficiaries.2-6 CMMI expressed special interest in demonstrations that integrate LTSS with other health care services, including behavioral health.
Massachusetts officials welcomed this opportunity. According to a 2011 report, of the roughly 105 000 dually eligible Massachusetts residents aged 21 to 64 years in fee-for-service Medicaid (MassHealth) plans, 79% had physical, 65% had behavioral, and 14% had developmental diagnoses or disability, and 60% had more than 1 of these conditions.7 Although only 3% of the population lived in institutions, dually eligible individuals generated $1.3 billion in MassHealth and $1.2 billion in Medicare costs. The higher MassHealth expenses reflected substantial LTSS needs, which are covered by Medicaid but not Medicare: 35% of total spending went to LTSS compared with only 22% for inpatient services.7
In August 2012, Massachusetts was the first state to receive a memorandum of understanding under CMMI's financial alignment initiative, and state officials started designing One Care. This program is unique among CMMI's demonstrations in targeting fully dual eligible beneficiaries aged 21-64 (ie, all persons qualify for Medicare because of disability).4,8 One Care opted for dually capitated payment in which both Medicare and MassHealth capitate reimbursements to participating One Care plans. In this fully integrated care model, a unique feature involves requiring community-based LTSS coordinators to function independently from One Care plans.6,8
As reported elsewhere, the Massachusetts disability rights advocacy community participated actively with MassHealth officials in designing One Care:
Beneficiary engagement in the design and implementation of One Care has been a hallmark of the demonstration and has occurred at numerous levels. The state has well-developed disability and behavioral health consumer advocacy communities, and state leaders were open to actively engaging in discussions and some decision-making with these community-based consumer advocacy partners.6
To support their One Care advocacy, several disability rights organizations created the Disability Health Alliance (DHA), which in January 2013 published a mission statement articulating its goals for One Care. While recognizing potential benefits from One Care, DHA urged caution:
The Demonstration seeks to integrate our health and support services and their funding into a single system that will better meet our needs. Integration has the potential to improve our access to medical care and to long term services and supports while preventing unnecessary complications of disability and chronic illness. Integration also brings a serious risk that our needs will not be better met. If too much of the combined money is spent on medical services and too little on support services, then our health and independence will suffer. Capitated financing inevitably creates incentives to provide less care, which puts some of the state's most vulnerable individuals at risk of dangerous underservice.
DHA questioned whether standard quality metrics used to measure health plan performance would adequately capture disability advocates' concerns about One Care quality, especially relating to LTSS and implications for enrollees' quality of life. DHA argued that individuals with disability should assume leading roles in defining and measuring care quality, reflecting priorities relating to community-based living and LTSS.
Having persons with disability lead, design, and perform assessments to improve their own care could yield especially compelling benefits. Individuals with disability often require wide-ranging services across the full continuum of care, from complex tertiary care to LTSS. Persons with the lived experience of disability are best positioned to judge whether these diverse services are well integrated and meet their multidimensional needs, not only supporting independent community-based living but also maximizing health, wellness, and quality of life. Furthermore, despite efforts to improve measures of integrated care and LTSS quality,9,10 others—beyond DHA—also question whether common quality metrics adequately capture how health services affect well-being and quality of life for persons with disability.10-13
Given these considerations, DHA disability advocates sought funding from governmental and private sources to support the advocates conducting independent monitoring of care quality after One Care's October 1, 2013, start. However, those resources never materialized.
This PCORI project aimed to pursue DHA's goals of consumers leading evaluations of One Care.
Guided by the DHA mission statement, we aimed to test the effectiveness of consumer-driven quality information in improving the One Care experiences of English- and Spanish-speaking enrollees with serious mental illness or significant physical disability. Here, we report the results of a cluster randomized controlled trial (see the “Study Design” section) to compare the effectiveness of 2 informational interventions in a 12-month period:
We administered PDQ-S a second time, after the 12-month YES Health initiative concluded, and used difference-in-differences methods comparing the baseline and subsequent PDQ-S results as our outcome measure. Our major research question was the following: Compared with baseline PDQ-S results, does providing information defined and generated by consumers about care quality to PCPs and primary care practice managers improve enrollees' subsequent perceptions of their quality of care? We hypothesized that PCPs would use consumers' views to shape their quality improvement activities, thus producing changes important to and detectable by One Care enrollees.
As noted above, the DHA mission statement motivated and drove this project's goals and activities. Based on their active participation in One Care design,6 the disability rights advocates felt empowered to take charge of their own quality oversight and to interact with health care providers about improving their care. Over the course of this study, the interdisciplinary study team repeatedly returned to this mission statement for guiding principles when making critical decisions. The project's principal investigator has a significant physical disability; she co-led the study with 2 disability rights advocates, 1 with significant physical disability and the other in recovery from a psychiatric diagnosis (both were salaried staff of the study). In addition, the project team solicited input at twice-yearly meetings (and as required between meetings) from a research oversight committee (ROC), which included local researchers with expertise in LTSS and clinical care for persons with disabilities, as well as representatives from physical disability and serious mental illness advocacy organizations. To obtain additional advice, we empaneled a 6-person Consumer Analysis Team (CAT), including 3 members with physical disability and 3 with serious mental illness. All collaborators received either salary support or consultation payments for their contributions. Persons with disability are coauthors of papers from this project and have participated in other dissemination efforts.
Our review of the literature failed to find any other completely consumer-led quality measurement initiative like YES Health. Therefore, we view YES Health as an innovative effort in which consumers themselves identified the quality concerns; collected and analyzed data about care quality; and gave this information to primary care practices and PCPs. Another important and singular feature of YES Health was the self-identification of the consumers leading this effort as individuals with significant physical disability or in recovery from serious mental illness.
This study tested the effectiveness of consumer-driven quality information to improve One Care experiences of English- and Spanish-speaking enrollees with serious mental illness or significant physical disability (see the “Project Overview” section). This section describes 3 critical components of this study: the conceptual framework developed by disability advocates that models the relationship between health care and health, wellness, and quality of life for persons with disability (see the “Conceptual Framework Guiding Consumer-Driven Quality Assessments” section); development of the PDQ-S, used to capture baseline perceptions of One Care quality as well as intervention outcomes (see the “Designing the Persons With Disabilities Quality Survey” section); and development and description of YES Health, a consumer-led initiative to assess One Care quality that reflected priorities of One Care members with serious mental illness or significant physical disability (see “YES Health” in the Background section). As detailed in the “Study Design” section, this cluster randomized controlled trial randomly assigned 27 primary care practices with ≥50 One Care members to 3 study arms differing by information provided to practice directors and PCPs: (1) quarterly YES Health reports plus results from baseline administration of PDQ-S to 720 enrollees before YES Health implementation; (2) baseline PDQ-S results only; and (3) no study information.
As described in the “Perspectives of Massachusetts Disability Rights Advocates” section, the DHA mission statement questioned whether standard quality metrics used to measure health plan performance adequately capture concerns of persons with disability about One Care quality, especially relating to LTSS and quality of life. DHA argued that individuals with disability should assume leading roles in defining and measuring care quality, prioritizing factors related to community-based living and LTSS. Therefore, to develop our 2 informational interventions—reports about One Care quality from data gathered using PDQ-S and YES Health—we first conceptualized how persons with disability perceive the role of health care.
Two disability rights advocates codirecting the project helped formulate a conceptual model depicting how health care and other factors affect the health, wellness, and quality of life of individuals with disability (Figure 1). As described in detail elsewhere,14 we reviewed the literature on care quality for persons with disability or with LTSS needs.
Conceptual Framework: How Health Care and Other Factors Affect the Health, Wellness, and Quality of Life for Individuals With Disabilities.
This review convinced us not to replicate other efforts. Four major international and US initiatives had developed specific components, with mutually supportive and overlapping tenets, that we fit together to build our conceptual framework:
Figure 1 pulls together these 4 overlapping components; at the center are the interrelated concepts of health, wellness, and quality of life. Personal factors of individuals with disability are grouped largely within categories from the ICF. Attributes of the health care delivery system and larger environment include physical features and health care professional workforce factors, including stigmatizing attitudes. Arrayed above interactions between individuals and the health care system, communication is broadly construed and encompasses not only culturally competent communication but also access to information, knowledge about and having options for care, shared understanding between persons and care providers, trust, and hope. This model informed our examination of the 12 surveys chosen by the Centers for Medicare & Medicaid Services (CMS) and MassHealth to evaluate One Care plan performance.8,21-32 The surveys determined whether, from the perspective of individuals with disability, gaps existed in quality assessment.14
In Year 1, we designed the PDQ-S, which served 2 roles in this project: First, we reported information generated by baseline (ie, before the start of YES Health) administration of PDQ-S to primary care practice managers and PCPs as 1 of our 2 informational interventions (see the “Reports of Baseline [Wave 1] PDQ-S Findings” section); and second, we used subsequent results from administering PDQ-S after the 12-month YES Health initiative in difference-in-differences analyses to assess outcomes and effectiveness of the 2 interventions (see the “Identifying the Wave 2 PDQ-S Sample,” “Study Outcomes”, and “Analytical and Statistical Approaches” section). Here, we describe the development and content of PDQ-S, which is detailed elsewhere.33 Project team members with physical disability or serious mental illness played leading roles in PDQ-S development and testing, with additional input from ROC and CAT members with disability (see the “Role of Persons With Disability in the Design and Conduct of This Study” sections), supporting its content validity.
Guided by our conceptual framework (Figure 1), we began by reviewing the content of the 12 surveys.14 This review identified gaps in these 12 quality measures from the perspective of persons with disability. We also examined existing measures of self-efficacy, which we viewed as related to independent living principles. These reviews provided information about how existing surveys approach certain concepts, problematic phrasing of questions, and other insights into designing surveys for this population. We developed PDQ-S in 4 phases (described below).
We conducted fourteen 30-minute telephone or in-person interviews with key informants—namely, individuals representing major One Care stakeholders. These interviews aimed to obtain participants' views about critical quality considerations confronting One Care, particularly for enrollees with serious mental illness or significant physical disability and matters relating to LTSS. To guide these interviews, we developed an 8-item, semistructured, open-ended interview protocol based primarily on DHA mission statement observations about measuring quality in fully integrated service delivery programs for persons with disability. Three interviews involved more than 1 interviewee (18 total key informants). Key informants included 7 disability advocacy stakeholders (including persons with lived experiences of serious mental illness, significant physical disability, or substance use); 2 representatives of the local union serving personal care assistants; 3 clinical leaders of One Care plans; 4 Massachusetts officials (rehabilitation commissioner, director of MassHealth quality measurement) and local health policy experts; and 2 CMS officials (1 each from CMMI and MMCO). Details differed, but interviewees provided largely consistent messages about which quality concerns would most affect One Care members with serious mental illness or significant physical disability. We appeared to reach thematic saturation approximately halfway through our scheduled interviews. Major themes included the following:
Several interviewees explicitly mentioned including Maslow's hierarchy of needs within a conceptual framework for One Care evaluation, with assessments along Maslow's 5-level hierarchy—from physiological needs to safety, to love and belonging, to esteem, and finally to self-actualization.
Building on key informant findings, we sought input directly from persons with serious mental illness or significant physical disability through focus groups. We designed a semistructured focus group moderator's guide through an iterative process, drawing on topics raised by the key informants, DHA's mission statement, and conceptual framework perspectives. Eligibility criteria included aged 21 to 64 years, self-reported disability (serious mental illness or significant physical disability), and insurance coverage (One Care or MassHealth). Participants received a $50 gift card for participation. Collaborators with disability led focus-group recruitment and screening activities across Massachusetts. A professional moderator facilitated all 12 focus groups, while an individual representing the lived experience with concordant gender and disability status comoderated each focus group. Analyses of transcripts aimed to identify potential domains and specific themes to inform PDQ-S development. A research consultant to the collaborating advocacy organization, led in-depth analyses conducted by staff with disability and CAT members.
With 87 participants, we conducted 12 gender-concordant and disability-concordant focus groups across Massachusetts: 4 in Spanish and 8 in English. Along with potential questions within each theme, 7 broad themes emerged from the focus group analysis. The analysis generated 38 questions for persons with serious mental illness and 29 questions for persons with significant physical disability. Broad themes were similar across the 2 disability groups, although the potential questions differed somewhat. Certain themes hewed closely to concepts within Maslow's hierarchy, such as love/belonging and self-esteem.14
PDQ-S development involved extensive, iterative input from all members of the project team, including the investigators, community partners, ROC, and CAT. A senior survey scientist (K.D.) led efforts to draw potential survey domains from information gleaned from the reviews of surveys used by CMS and MassHealth to evaluate One Care, the key informant interviews, and focus groups. Our initial set of domains aimed to (1) assess how One Care members with serious mental illness or significant physical disability perceive core components of One Care, and (2) evaluate effects of One Care on the daily lives and well-being of its members. We sought to create a survey that would average 10 to 15 minutes to complete.
The draft PDQ-S encompassed 6 domains that cut across One Care's service delivery model, including LTSS, and participants' perceptions of their daily lives and how One Care affects them. A certified translator translated PDQ-S into Spanish. To be inclusive for the testing phase, the draft PDQ-S was substantially longer than desired for the final instrument.
We tested the draft PDQ-S using cognitive interviews: a semistructured, open-ended protocol with 8 debriefing questions, in English and Spanish. The debriefing questions identified items needing revision (eg, because the topic was too sensitive, because wording was unclear). Interviewers timed the interview to determine the length of survey administration. With training and oversight from a survey scientist, 3 staff with serious mental illness or significant physical disability from the Disability Policy Consortium (DPC; Malden, MA) conducted the respondent recruitment, screening, and cognitive interviewing. The 20 participants (10 each within each disability group) each received a $50 gift card. After completing the cognitive interviews, the DPC interviewers and survey scientist analyzed responses to the debriefing questions, summarized key findings, and proposed changes to the PDQ-S. Project team members reviewed these suggested revisions and made extensive comments. PDQ-S was finalized based on iterative discussions, with ultimate choices (eg, about which questions to retain or delete) made by team members with disability.
Table 1 summarizes content of the final PDQ-S. It contains 15 substantive items in addition to an introductory item confirming respondents' enrollment in One Care, 11 sociodemographic characteristic queries, and a final item for contact information (used to mail the incentive payment).
Main Questions Included in Final Version of the PDQ-S.
To operationalize the vision of the DHA mission statement—persons with disability proactively measuring their own quality of care and engaging with providers to improve care—DPC staff members developed for this study a multifaceted initiative that they called YES Health: Your Experience, Speak up for better health care. English- and Spanish-speaking DPC staff members, all of whom self-identify as having a physical disability or being in recovery from a psychiatric diagnosis or serious mental illness, designed and implemented YES Health. Their goal was to engage One Care enrollees with serious mental illness or significant physical disability in reporting about their care quality. DPC staff would then compile these quality assessments and communicate them directly to PCPs caring for One Care members, with the intent of improving that care. YES Health, which DPC staff members conducted over a 12-month period, was 1 of the 2 informational interventions assessed in this cluster randomized trial (study arm 1).
DPC staff conceptualized YES Health on the principles of community-based participatory research (CBPR), in which activities aim ultimately to motivate change and foster empowerment and capacity within communities.34-38 In particular, YES Health built on notions of “empowered consciousness” and “nothing about us without us.”37,39 YES Health pushed beyond usual CBPR precepts, using an “about us, by us” framework. Instead of looking outside the community to synthesize scattered ideas and implement research, YES Health sought leadership, expertise, and partnerships within diverse disability communities. DPC staff members led YES Health development and implementation, while the researchers primarily provided logistical support and technical advice as specifically requested by DPC staff for building their methodological capacity.38
DPC staff spent approximately 6 months designing YES Health concepts and its multiple components (Table 2). Based on responses to the request for proposals, DPC selected the Health Communication Core (HCC) at Dana-Farber Cancer Institute to support design of the intervention's communication strategies and its website. HCC personnel met at length with DPC staff to delineate details about the goals, general tone, and components of the initiative; a specific “brand image” that would be consistent across all communication modalities; and technical aspects of the website, particularly disability access. HCC proposed several naming options to brand the initiative, including YES Health, which DPC chose. DPC staff identified technical website disability accessibility standards to guide HCC programmers. DPC personnel made final decisions about all written content and appearance of YES Health materials. Resource limitations prevented us from generating new photographs and extensive graphics to illustrate the website; instead we used low-cost stock images and sought a bright, positive color scheme.
Components of the YES Health Initiative.
During the 3 months of YES Health website programming and testing, DPC staff focused on other aspects of YES Health (Table 2), including specifying enrollment and eligibility criteria for YES Health membership (a key criterion was being assigned to 1 of the 9 primary care practices in study arm 1); creating the membership screening process; establishing telephone, voice mail, and email procedures; designing postcards, rack cards, and flyers advertising YES Health; establishing a YES Health Facebook group; and developing procedures for data gathering using Survey Monkey (via website or telephone). The 2 lead DPC staff members wrote and appeared in an introductory video, which HCC produced and posted on the YES Health website. To educate local disability communities about YES Health—and obtain input about ways to recruit One Care enrollees to join YES Health—DPC staff visited advocacy organizations statewide near the 9 primary care practices. They also hired YES Health “ambassadors” from the community. Although limited resources prevented us from creating a parallel Spanish-language version of the website, other communication materials (except the postcards; see Table 2 footnote) were made available in Spanish. DPC staff who are native Spanish speakers reached out to advocacy organizations serving Hispanic populations, responded to all Spanish-language voice mails, and assisted Spanish speakers in answering the quarterly surveys.
YES Health aimed to gather data quarterly about issues that especially concerned One Care members with significant physical disability or serious mental illness. DPC staff chose the quarterly topic; drafted roughly 8 to 12 questions addressing critical concerns for each topic; sought feedback from local disability advocates; programmed the final questions into Survey Monkey; conducted the survey, including helping participants answer by telephone; and used basic descriptive methods to compile and analyze the results. DPC staff summarized these findings in quarterly reports sent to PCPs (see the “Quarterly Reports of YES Health Quality Assessments” section) and YES Health members.
The major research question was the following: Does providing consumer-defined information about consumers' quality of care to practice managers and PCPs improve the perceptions of One Care members with serious mental illness or significant physical disability of their quality of care? A second research question was the following: Does the nature and intensity of the consumer-defined quality information affect the extent and type of quality improvements, again from consumers' perspectives? We designed our study as a cluster randomized controlled trial with the primary care practice as our unit of randomization.
We randomly assigned 27 participating primary care practices with ≥50 One Care members with significant physical disability or serious mental illness to 1 of 3 study arms, which differed by the type of information mailed to providers:
We administered PDQ-S again 1 year later (wave 2, after completion of the 12-month YES Health initiative) and used difference-in-differences analyses of PDQ-S results across the 2 waves to assess intervention outcomes.
As described in the “Policy Context” section, the study's setting was the Massachusetts One Care demonstration program for individuals aged 21 to 64 years who are dually eligible for Medicare and full Medicaid (MassHealth) benefits.4,6,8 We submitted the proposal to PCORI in August 2013. During that August, CMS, MassHealth, and the 3 selected One Care plans (which received final contracts in mid-July 2013) were urgently finalizing arrangements for One Care's launch.6 The 3 plans—Commonwealth Care Alliance (CCA), Fallon Total Care (FTC), and Network Health—covered all Massachusetts counties except Barnstable, Berkshire, Bristol, Dukes, and Nantucket (Figure 2). All 3 plans provided letters of support indicating likely willingness to participate in the PCORI study, should it be funded.
Massachusetts One Care Coverage Map as of Fall 2013.
One Care began operating on October 1, 2013. The early implementation of One Care6,8 confronted significant logistical and financial challenges.6,8 Network Health had other priorities and declined participation in the PCORI project. FTC participated but announced plans to leave One Care just as we were starting baseline (wave 1) PDQ-S administration (late spring 2015). Therefore, the remaining plan, CCA, was our study setting. CCA is a private, nonprofit, hybrid health plan and health care delivery system with a long history of providing fully integrated care to low-income persons with heavy burdens of disease and disability.8,40-44 It serves the largest geographic area of all plans (Figure 2). CCA had substantially larger numbers of One Care members than did the other 2 plans. For our randomization purposes, CCA had 27 practices caring for ≥50 One Care members with significant physical disability or serious mental illness.
This section describes selection of participants in the wave 1 PDQ-S administration, the “Interventions” section describes the study cohort for the YES Health intervention, and the “Identifying the Wave 2 PDQ-S Sample” section describes selection of participants for wave 2 PDQ-S administration.
Wave 1 PDQ-S involved a sample of 720 community-dwelling adults with significant physical disability or serious mental illness who were enrolled in CCA's One Care program (ie, persons assigned to MassHealth rating categories C2A, C2B, C3A, and C3*). All eligible individuals had been enrolled in CCA's One Care for ≥90 days. Each sample member was a patient in 1 of the 27 primary care practices staffed or contracted by CCA that were eligible for our study. Eligible practices had ≥50 One Care enrollees with significant physical disability or serious mental illness. We drew the survey sample of 720 members from a universe of 3483 CCA enrollees in those 27 practices whose coverage by CCA started before May 2015. We excluded individuals from the wave 1 sample if CCA's records indicated that they spoke a language other than English, Spanish, or American Sign Language.† We randomly selected roughly equal numbers of enrollees (25-27) from each practice (a stratified random sample where the strata of interest were practices). We selected 50 enrollees from 1 practice because it had many more enrollees than did the other 26 practices. Although about 21% of the sample members did not have a designated PCP specified in the sampling frame, all enrollees in the universe of eligible enrollees did have a designated primary care practice among the 27 eligible practices.
We created a practice-level report of baseline (wave 1) PDQ-S findings that included a cover letter describing the survey and a 3-page report of text and graphics showing practice-level data, as well as comparative data from a random sample of all wave 1 participants. We sent these reports to the practice manager at each of the 18 practices randomized to the 2 intervention arms (arms 1 and 2). Approximately 1 month later, we sent the same practice report to 221 PCPs that the health plan indicated served as the PCP for at least 1 individual in the wave 1 sample. One very large practice had 24 eligible PCPs; the others averaged 7 PCPs per practice (range, 4-14). Figures 3 to 8 show examples of tables and figures sent to PCPs; the Results section presents details about wave 1 PDQ-S results. We invited questions or follow-up from the practices and received only 1 anecdotal report from a provider that she had seen the data.
Confidential Summary.
One Care Team and Plan.
Independence.
Primary Care Experiences.
One Care Experiences.
Overall Assessment.
To prepare for sharing quarterly YES Health quality assessment reports with practice managers and PCPs at the 9 study arm 1 primary care practices, DPC staff contacted practices and offered to describe YES Health to practice staff either in person or by telephone. DPC staff sought advice about the most effective method to provide the YES Health quality assessments to PCPs so that PCPs would engage with them and support quality improvement activities. DPC had considerable difficulty getting responses from the practices despite multiple and varied attempts at contacting different practice personnel. Without input from the PCPs or practice managers, the most efficient and straightforward approach for distributing the quarterly reports was by regular US Postal Service mail (CCA had provided PCP and practice mailing addresses but not email addresses) in the form of a 2-page report with both narrative and numerical content. DPC staff conceptualized, produced, and mailed these quarterly reports to each PCP and the practice manager. DPC staff sent similar content in written newsletters to YES Health participants.
Figures 3 to 8. Reports About Baseline PDQ-S Findings for study arms 1 and 2: Examples of Tables and Figures Sent to Practice Managers and PCPs
We administered the wave 2 PDQ-S 12 months after wave 1 and as the YES Health intervention concluded. As in wave 1, the wave 2 PDQ-S included a sample of 720 community-dwelling adults with significant physical disability or serious mental illness who are enrolled in CCA One Care and are assigned to 1 of the 27 study primary care practices in the 3 study arms. In preparation for wave 2, we compared the list of all eligible One Care enrollees from which the wave 1 sample was drawn with an updated file of CCA One Care enrollees active as of April 2016 and determined that more than 600 of our wave 1 sample was still enrolled and eligible for wave 2. In consultation with study leaders and their statistical consultant, Mathematica retained wave 1 sample members who were still enrolled in CCA One Care and had no PCP change recorded or who were still enrolled in One Care but had changed to a PCP in another eligible study practice. We excluded wave 1 enrollees who changed to a PCP outside of the 27 study practices and disenrollees from CCA. We replaced excluded or disenrolled sample members with other eligible enrollees on a practice-by-practice basis consistent with the number of enrollees who had left that practice.
We hypothesized that gathering consumer-defined quality information from persons with disability and reporting this information to PCPs would motivate PCPs to improve their quality of care as perceived by One Care enrollees. Wave 2 PDQ-S results would indicate postintervention perceptions of care quality of One Care enrollees with significant physical disability or serious mental illness. The change between wave 1 and wave 2 PDQ-S results would indicate whether either or both interventions (see the “Reports of Baseline [Wave 1] PDQ-S Findings” and “Quarterly Reports of YES Health Quality Assessments” sections) affected the perceptions of One Care enrollees—and thus suggest whether PCPs had improved their care.
The “Reports of Baseline (Wave 1) PDQ-S Findings” and “Quarterly Reports of YES Health Quality Assessments” sections describe data collection and sources for the 2 informational interventions (report of baseline PDQ-S findings and quarterly YES Health reports). Here, we describe data collection for PDQ-S waves 1 and 2. We administered PDQ-S twice, 1 year apart, each during 12 to 13 weeks in June-September 2015 and 2016. Our multimodal design included self-administered web and mail modes and interviewer administration by telephone.‡ This design gave sample members the opportunity to respond using the mode and time frame most convenient for them. In each wave, we sent all eligible participants an advance letter with information to complete PDQ-S online. Persons who completed PDQ-S in the first 10 days of the field period received a $15 gift card, while those completing it later received a $10 gift card. About 27% of survey respondents competed PDQ-S within the 10-day window. The distribution of chosen administration modes was similar in both waves. Of the 3 possible modes, participants preferred telephone administration (69.5%). Of the small percentage of respondents who answered by web in wave 1 (5%), only a few (10% of the 5%) also participated by web in wave 2.
We calculated response rates as 62.7% in wave 1 and 60.6% in wave 2; weighted response rates were 65.4% and 62.0%, respectively, accounting for selection probabilities. We calculated response rates using Response Rate No. 4, American Association for Public Opinion Research (AAPOR) 2016.§ As such, both waves met our target response rate of 60%. Most importantly, we succeeded in including populations that can be difficult to reach, such as people who have experienced homelessness and those who speak only Spanish. Throughout the field period, Mathematica monitored data collection for the sample overall and by subgroup, including tracking survey completion across the 27 practices, the 3 study arms, language of survey completion, and the 2 subgroups of disability (significant physical disability and serious mental illness). At the close of the field period, response rates ranged from 37.5% to 75.0% across the 27 practices, and were 62.1%, 60.5%, and 57.2% by study arm, respectively. Final response rates were 63.8% for persons with physical disability and 54.6% for individuals with serious mental illness. Response rates among Spanish (66.7%) and English speakers (63.3%) were similar (persons with unknown language preferences had a 51.8% response rate).
We had originally intended to conduct semistructured, open-ended interviews with 2 individuals at each of the anticipated 12 YES Health practices (see the “Conduct of Study” section); however, the lack of responsiveness from practice personnel (see the “YES Health Results” section) caused us to change our plans.
To elicit feedback directly from PCPs to whom we had sent reports, we developed a brief survey to be mailed to the PCPs of record for the enrollees we had sampled for PDQ-S wave 1. Survey development proceeded in 4 phases: review of key informant interviews conducted with physician stakeholders at the start of the project; solicitation of feedback from the research team and ROC; instrument drafting by our survey research methodologist; and final review by 2 other survey researchers with expertise in clinician surveys.
Because the reports we had sent to the PCPs included PDQ-S findings, we sought to assess the value that PCPs placed on PDQ-S-type information and test their knowledge of key elements of One Care and quality reporting about One Care. Survey domains included attitudes about the integration of physical and behavioral health care for persons with disability (3 items); items about One Care plans, care teams, and new services made available to patients through One Care (identical to measures in PDQ-S; 5 questions); provision of primary care services important to persons with disability and emphasized in YES Health reports (3 items); recall of quality reports, including PDQ-S and YES Health (4 items); an overall assessment of benefits of One Care for patients (5 items); and provider characteristics (4 items). We conducted the survey by postal mail among the PCPs assigned to the enrollees sampled in wave 1 of the PDQ-S. During October to December 2016, we mailed surveys to PCPs, enclosing $50 in retail gift cards. We sent up to 2 more mailings to nonrespondents, correcting address errors or changes through outreach to practice administrators, web searches, and further updates from CCA. If we could not locate the targeted physician in Massachusetts or in one of our study practices, we excluded them from the eligible physicians for this survey.
Among the 221 PCPS listed by the health plan as the providers of record for wave 1 sample members, 19 were excluded for having subsequently left an eligible practice, and 10 refused participation. We received surveys from 110 of 202 eligible PCPs (54.4%), although 7 indicated they had no One Care patients and therefore could not answer questions about One Care enrollees. We nonetheless included them in our analyses.
We focus here on the analysis of the wave 1 and wave 2 PDQ-S data; the differences in results from these 2 surveys represent our main outcome measure (ie, whether enrollees' perceptions of quality of care, broadly defined, have changed across the 12-month interval between waves; see the “Study Outcomes” section). We performed descriptive analyses of cohort characteristics and cross-sectional PDQ-S results within waves, and longitudinal analyses across the 2 PDQ-S waves. We compared the change from wave 1 to wave 2 and among the 3 within each wave, using chi-square tests. We also analyzed results from the randomized trial (see the “Study Design” section) by comparing the changes from wave 1 to wave 2 among the 3 study arms by testing the wave and arm interaction in the logistic regression models. We performed a subanalysis using only results from persons who completed both wave 1 and 2 surveys. We conducted all analyses using SAS survey procedures to consider survey sampling structure and sampling weights. We report weighted percentages in the Results section to reflect the underlying populations.
We conducted primarily descriptive analyses of the PCP survey. We also ran analyses by PCP assignment to each of the 3 study arms. These analyses were affected by small numbers of PCPs in certain combinations of study arms and question response category.
As in any randomized trials with surveys, our study has 2 key assumptions: (1) The study groups are similar; and (2) there is no nonresponse bias. For the first assumption, we found some imbalances across the groups because randomization occurred at the practice level, not at the patient level. Therefore, we conducted regression analyses including these patient-level characteristics in the model to account for these imbalances. For the second assumption, we adjusted the original sampling weights assigned to each individual to account for differences between respondents and nonrespondents.
As stated in the “Key Assumptions of Methods” section, we accounted for nonresponse by adjusting the original sampling weights assigned to each individual. For each adjustment factor that we calculated, the goal was to increase weights of interviewed individuals who are most similar to those individuals who could not be interviewed, thereby counteracting differences between respondents and nonrespondents and reducing the potential for nonresponse bias.** For specific PDQ-S items, missing data ranged from 0% to 8% with a mean of 2.7%. With such small amounts of missing information, we did not attempt to impute values.
We examined the heterogeneity of treatment effects in subgroups identified by type of disability (significant physical disability vs serious mental illness). Our (null) hypothesis is patients with physical disability and serious mental illness will report similar overall One Care experiences and perceptions in responding to PDQ-S; however, we also hypothesize that the 2 groups will differ somewhat in responses to individual PDQ-S items.
As documented throughout this project, all aspects of the study were approved by the Partners Human Research Committee IRB. We submitted multiple amendments to the IRB as the study proceeded, addressing each upcoming activity in turn.
As described in the “Study Setting” section, the major deviation of our final study from the proposed protocol involved the participation of a single One Care plan. We originally intended to include all 3 One Care plans and to randomize 36 practices to the 3 study arms (12 practices from each One Care plan). However, with CCA as our sole participating plan, we had only 27 practices that met our criteria for participation (ie, ≥50 One Care members with either significant physical disability or serious mental illness).
The other major deviation from our proposed plans resulted from the difficulties engaging PCPs and practice managers through the YES Health initiative (see the “Survey of Primary Care Physicians in Year 3” and “YES Health Results” sections). That led us to conduct a survey of PCPs in months 32 to 33 of the project.
In this section, we present our results as follows: description of study population (see the “Description of Study Population” section); results from the baseline (wave 1) PDQ-S, which was sent to practice managers and PCPs in study arm 1 and 2 practices (see the “Baseline [Wave 1] PDQ-S Results” section); findings from the YES Health quarterly survey (ie, information sent to practice managers and PCPs in study arm 1; see the “YES Health Results” section); results of the randomized controlled trial (see the “Results From Cluster Randomized Controlled Trial” section); and findings from the PCP survey (see the “Physician Survey Results” section). CONSORT diagrams describing the administration of our 3 surveys, PDQ-S wave 1, PDQ-S wave 2, and the PCP survey, can be found in Figures 9, 10, and 11, respectively.
CONSORT Diagram for Wave 1.
CONSORT Diagram for Wave 2.
CONSORT Diagram for Primary Care Physician Survey.
In wave 1, a total of 451 persons answered PDQ-S: 160 in the YES Health and PDQ-S arm; 156 in the PDQ-S-only arm; and 135 in the control arm. In wave 2, a total of 426 persons answered PDQ-S: 154 in the YES Health and PDQ-S arm; 141 in the PDQ-S-only arm; and 131 in the control arm. As noted in the “Key Assumptions of Methods” section, we randomized practices not participants; therefore, imbalances across study arms did occur with certain participant attributes. Table 3 shows demographic characteristics of participants in each of the 2 waves across the study arms.
Sociodemographic Characteristics by Study Arm and Survey Wave.
In wave 1, participants across the 3 study arms were similar by age, gender, disability type, education, language of survey completion, current living arrangements, and whether persons had experienced homeless in the past 6 months. Across all 451 participants, 45.8%†† were male, 37.8% had significant physical disability, 33.9% had less than a high school education, 83.9% completed the survey in English, 42.6% were living alone, and 10.5% had experienced homelessness in the past 6 months. In contrast, significant differences appeared across the 3 study arms in other participant characteristics. For example, overall 39.3% participants were White; however, 54.1% of participants in the YES Health/PDQ-S arm were White, compared with 25.2% in the PDQ-S-only arm and 37.8% in the control arm. Marital status also differed significantly: Overall, 44.9% participants were single, but 31.3% of participants in the YES Health/PDQ-S arm were single, compared with 51.4% in the PDQ-S-only arm and 53.4% in the control arm.
In wave 2, distributions of participants by age, gender, language of survey completion, current living arrangements, and whether persons had experienced homeless in the past 6 months did not differ statistically significantly across the 3 study arms. Across all 426 participants, 46.6% were male, 83.9% completed the survey in English, 38.9% were living alone, and 9.2% had experienced homelessness in the past 6 months. Significant differences appeared across the wave 2 study arms in other participant characteristics. Most importantly, overall, 41.0% participants had physical disability; however, 36.7% of participants in the YES Health and PDQ-S arm had physical disability, compared with 34.6% in the PDQ-S-only arm, and 53.2% in the control arm. Educational status also showed significant differences: Overall, 31.5% participants had less than a high school education, but 28.2% of participants in the YES Health and PDQ-S arm had this level of education, compared with 24.6% in the PDQ-S-only arm, and 43.4% in the control arm.
Between waves 1 and 2, the only statistically significant difference in distribution of demographic characteristics among PDQ-S participants involved living arrangements in the control group: Persons in wave 1 were much more likely to live alone (45.4%) than those in wave 2 (32.2%).
Results from wave 1 PDQ-S administration appear in Table 4 (perceptions relating to independent living principles), Table 5 (perceptions of One Care and PCP experiences), and Table 6 (perceptions of daily life). We used these results to produce the baseline reports sent to practice managers and PCPs in study arms 1 and 2 (Figures 3-8). We describe these results further—in comparison to wave 2 findings—in the “Results From Cluster Randomized Controlled Trial” section. For purposes of the randomized clinical trial, it is important to note that wave 1 findings did not vary significantly across the study arms for independent living principle questions (Table 4) and perceptions of One Care and PCP experiences (Table 5). In wave 1, the only statistically significant difference in responses across study arms involved the percentage of persons saying their relationships with other people were better now than in the past year: 25.1% in study arm 1 compared with 40.2% in arm 2 (Table 6).
Perceptions Relating to Independent Living Principles by Study Arm and Survey Wave.
Perceptions of One Care and PCP Experiences by Study Arm and Survey Wave.
Perceptions of Daily Life by Study Arm and Survey Wave.
Over the 1-year implementation, DPC screened 112 individuals, and 45 (40.2%) were eligible and joined YES Health (Table 7). This represents 3.6% of 1255 potential participants in the 9 YES Health practices identified by Mathematica from the CCA database. DPC had spent considerable time and resources in developing its website and online communication approaches; however, most YES Health members (32/45 [71.1%]) preferred to receive routine communications (eg, newsletters) through regular US mail rather than online (Table 7). Similarly, most (37/45 [82.2%]) preferred to complete the Survey Monkey quarterly survey by telephone with DPC staff rather than online. Although 29 (64.4%) had used Facebook previously, only several participants joined the YES Health Facebook group discussions, which DPC stopped staffing after 8 months (Table 2).
Characteristics of Persons With Disability Who Signed Up for YES Health.
Sequentially, the quarterly surveys of YES Health members considered (1) One Care care plans and care teams, (2) communication, (3) LTSS, and (4) transportation. DPC staff found that enrollees often did not understand even basic aspects of One Care, such as requirements for members to each have a written care plan and care team, which they participated in forming. Surveys therefore typically started by describing the topic (eg, care plans). Table 8 presents the questions and response categories from each YES Health survey, which aimed to take <15 minutes and could be completed either on the YES Health website or by telephone with English- or Spanish-speaking DPC staff. The transportation survey attracted the most respondents (36/45 [80.0%]).
Quarterly YES Health Member Survey: Survey Topics, Questions, and Responses.
DPC analyses included reviews of the narrative responses, present in roughly 50% of the open-ended questions. DPC summarized messages from the 4 surveys as follows:
Quarter 1, 3, and 4 survey results were summarized in newsletters to YES Health members.
DPC staff tried numerous times to contact either the medical director or the practice manager at the 9 YES Health primary care practices. One practice invited DPC to visit and suggested names of One Care enrollees who might be candidates for YES Health (4 subsequently joined). Another practice had DPC staff visit, but practice personnel questioned the methodological rigor of the PCORI study instead of discussing YES Health goals. By telephone, 1 practice representative requested not to be contacted again about YES Health. We had planned originally to engage productively with medical directors and/or practice managers in ongoing discussions about YES Health members' One Care experiences; however, DPC shifted its strategy to mailing a 2-page, quarterly report to 60 primary care providers listed by CCA on its enrollment files as serving One Care members with significant physical disability or serious mental illness at the 9 practices (Figure 12 shows the Quarter 4 report). In addition to rack cards (Table 2), the first mailing contained an introductory letter and 2 narratives (about a YES Health ambassador and a One Care member). For the other 3 reports, the front side contained a personal narrative, while the back side presented several summary results from the survey (Quarters 1, 3, or 4) and recommendations for clinicians from DPC staff based on survey findings. No clinicians or practice managers contacted DPC about these reports.
Fourth Quarterly Report to Primary Care Physicians From YES Health.
We analyzed changes in PDQ-S responses between waves 1 and 2 (Tables 4, 5, and 6) to assess whether providing YES Health and/or PDQ-S information to practice managers and PCPs affected perceptions of the quality of care among One Care members with significant physical disability or serious mental illness. With few exceptions, we failed to find any differences in PDQ-S responses between waves 1 and 2. Particularly, differences-in-differences analyses did not identify evidence that our interventions had improved enrollees' perceptions of their care quality. A subgroup analysis from the 315 persons who completed PDQ-S in both waves 1 and 2 (112 in study arm 1, 115 in arm 2, and 88 in arm 3) also failed to find evidence that our interventions had any effects.
Despite this disappointing outcome from the randomized controlled trial, PDQ-S, which was based largely on input from individuals with significant physical disability or serious mental illness,33 may offer insights not available from other widely used surveys.36 Descriptively comparing wave 1 and 2 PDQ-S results shows the absence of significant differences between the years but nonetheless highlights findings that suggest perceptions of socioeconomically disadvantaged persons with significant physical disability or serious mental illness enrolled in a highly integrated managed care program. We present findings by broad domains of PDQ-S in the “YES Health Results” section: perceptions relating to independent living principles, perceptions of One Care experiences and PCPs, and perceptions of daily life.
Table 4 shows results of 5 questions relating to principles of independent living. We found no significant differences between responses in wave 1 compared with wave 2. Across all 3 study arms we found the following responses to PDQ-S questions:
Table 5 shows results from several questions about key components of One Care and members' perceptions about their PCPs.
All members of One Care are required to have a care team, which the member helps identify. Across all survey respondents, 61.0% in wave 1 and 63.5% in wave 2 reported having a care team. Similarly, all One Care members are required to have care plans, which they help develop and of which they are given a copy. Overall, in wave 1, 55.7% reported having a care plan, 48.8% indicated they helped develop the plan, and 30.2% said they had a copy of the plan. In wave 2, 59.1% reported having a care plan, 52.4% indicated they helped develop the plan, and 32.1% said they had a copy of the plan. Only the PDQ-S-only study arm showed significant improvement between wave 1 and wave 2: 66.6% reported having a care plan in wave 2, compared with 52.1% in wave 1 (P = .022); 60.1% indicated they helped develop the plan in wave 2, compared with 44.0% in wave 1 (P = .012).
As shown in Table 5, we found no differences between waves 1 and 2 in how persons felt about their PCPs. Across all 3 study arms, we found the following responses to PDQ-S questions:
“Since you enrolled in this health plan, about how often do you feel your primary care provider…”
Respondents of neither disability type reported significant changes on experiences with PCPs between wave 1 and wave 2.
As shown in Table 6, we found few significant differences across waves in responses to the 9 questions about daily life. We found the following responses to PDQ-S questions:
“Since enrolling in your health plan, is your life now better, the same, or worse in the following areas?”
“Would you say the quality of all the health care that you have received is better than before, about the same as before, or worse than before?”
As described in the “Survey of Primary Care Physicians in Year 3” section, the failure of YES Health to engage PCPs raised concerns that the randomized controlled trial might fail: If physicians were unaware of the intervention, how could the intervention succeed? We therefore conducted a survey of PCPs assigned to the One Care enrollees sampled in wave 1 of PDQ-S. As shown in Table 8, some respondents failed to report demographic information among the 109 responders: 12.8% with age missing, 14.7% with gender missing, 14.7% with race missing, and 15.6% with ethnicity missing. Among those who provided information, across the study arms the mean (SD) age was 49.0 (10.6) years; 45.8% were male; 79.8% were White; and 9.7% were Hispanic.
Table 9 shows responses to questions about their One Care experiences, their perceptions of the program, and exposure to study materials. Of the 48 physicians assigned to practices in the YES Health arm, only 2 indicated having seen YES Health materials, which had been mailed to them quarterly. Of these 48 physicians plus the 39 physicians assigned to practices in the PDQ-S-only arm, only 6 reported having seen the PDQ-S reports, which had been mailed to them roughly 9 months previously. Notably, 38 (34.9%) physicians were not sure whether their One Care patients had a formal care team, as designated by the program. Fifty (45.9%) said they had not worked with their patients to create a care plan, as also required by the program. Most were not sure about how One Care had affected their patients' lives.
Characteristics of Primary Care Physicians Responding to PCP Survey.
Responses to Primary Care Physician Survey.
YES Health suggests that persons with disability can lead evaluations of their care quality, focusing specifically on issues that particularly concern them about improving their health and quality of daily life. However, this research highlights 2 significant challenges: (1) difficulties engaging large numbers of persons within the disability community to participate in proactive quality monitoring; and (2) problems attracting PCPs to partner with and learn from their patients with disability, with the shared goal of improving care. Overall findings from PDQ-S suggest that One Care frequently functions as intended (eg, with enrollees participating in preparing their own care plans with care teams they designated) and improving enrollees' lives. Nevertheless, considerable room for improvement remains. Having motivated individuals with disability work collaboratively with physicians could potentially inform quality improvement efforts with powerful, person-centered insights.
As noted in external assessments of the Massachusetts One Care program, disability rights advocates made important contributions to designing One Care. “State leaders were open to actively engaging in discussions and some decision-making with these community-based consumer advocacy partners.”6 Buoyed by this involvement, Disability Health Alliance members—some of whom played leading roles in this study—believed they could proactively engage providers in improving quality of care. Therefore, in this study, the disability advocates developed and led YES Health with the long-term goal of building competence and capacity within their organizations for ongoing monitoring of One Care quality. Assessing whether YES Health could engage persons with disability in their own health care and promoting care improvement were important goals. YES Health succeeded in creating a structure, conceived and built by individuals with disability, which captured the voices of their intended audience: One Care members with significant physical disability or serious mental illness.
Nevertheless, before YES Health began, even the 2 DPC project leaders had difficulty predicting how many One Care members would join. This target audience has significant disability, as judged by MassHealth rating categories; many of them also have comorbid conditions, including substance abuse disorders, further compromising their health. In addition, all are sufficiently financially impoverished to receive full Medicaid benefits. In this context, attracting 45 members, conducting and analyzing 4 surveys, and mailing reports summarizing findings to primary care practices and providers are important accomplishments. However, as perhaps should have been anticipated,45 YES Health faced many impediments to achieving its goals, which may inform similar future efforts.
First, less than 4% of potential participants joined YES Health. We do not know why others did not. Nonetheless, we believe that making even $10 payments to members was critical to maintaining their engagement in both survey and town hall activities. We had initially aimed to attract participants by presenting this initiative as proactive collaboration between DPC staff, all of whom have disability, and community members with disability. DPC staff argued that voices of persons with disability typically go unheard or are interpreted—and thus distorted—by people outside the community. As 1 DPC staff member wrote, “Every victory for our rights and our dignity that our community has won came from people raising their voices together.” However, nearly all advocacy organizations that DPC staff visited to introduce YES Health highlighted the necessity to pay for participation. Throughout YES Health implementation, DPC staff heard from participants how they used these $10 payments (eg, to purchase milk). Providing monetary incentives for participation therefore seems necessary to motivate and continue participation among the populations like One Care enrollees.
Second, DPC staff spent considerable effort and resources creating and maintaining the YES Health website and Facebook group, yet few YES Health members used these online resources. Many participants reported not having access to computers or a way to connect online. YES Health members preferred telephone contacts, viewing personal conversations as building relationships. These calls with DPC staff generated rich information—beyond the survey—about members' health care experiences. However, all YES Health participants were aged 31 and older. Younger persons may have had different preferences about communication mode and gravitated toward online interactions. We therefore believe that moving forward, multimodal outreach—both online and in person—is essential to attract and communicate effectively with this population.
Finally, PCPs and practice managers showed no interest in YES Health. DPC staff received no response from mailing hundreds of 1-page reports, which included One Care member narratives, brief survey results, and recommendations for improving care. Many factors could explain clinicians' disinterest, including hectic schedules, disregarding postal mail, questioning YES Health methods and/or motivations, and unequal power dynamics between patients and clinicians.45 CCA was very supportive of this study; however, CCA wanted to limit outreach from the research team to their PCPs who were newly implementing One Care among their most complex enrollees. CCA informed PCPs about the study but did not require them to respond in any specific way. Given the resource and time constraints that squeeze many primary care practices in general, efforts to force participation among PCPs serving One Care enrollees (eg, by requiring certain quality improvement activities with community involvement) would have been unlikely to succeed—especially since there are no models for this type of consumer-directed interaction. Future efforts could focus on interactions with health plan leaders themselves rather than on their contracted or supported primary care practices.
Despite these difficulties, YES Health represents an innovative effort in which persons with disability led direct consumer evaluations of health care. YES Health activities did build capacity38 within the Massachusetts disability community for future consumer involvement. As 1 DPC staff person wrote,
Ultimately, we learned that by being people with disabilities ourselves, we provided an avenue where people felt comfortable opening up to us…sharing their own stories. Their stories were heard by people who at times had similar issues as the participants. This allowed for deep conversation and candid discussions about what people were experiencing on a day-to-day basis.
Some participants told DPC staff that, through YES Health, they felt more prepared to engage in their own health care. Although we cannot quantify whether those feelings improved health and wellness, this empowerment likely benefitted YES Health members through hearing and valuing their stories.
This was a negative study: We failed to find that providing consumer-generated data—through postal mailings of brief reports—improved care, as judged by reports from One Care members with significant physical disability or serious mental illness. However, the tools developed during this project, notably the PDQ-S and YES Health approach, might inform future quality measurement and health care advocacy for persons with disability. Through the advocacy channels of our DPC team members, we are attempting to disseminate these methods and use them for current advocacy. For example, in spring 2018, MassHealth planned to make major changes to move most members with disability into accountable care organizations. DPC is using the YES Health approach to gather information from the disability community about its members' perceptions of these changes and what advocacy activities would best assist them.
YES Health may have limited generalizability. The initiative involved Massachusetts One Care members with significant physical disability or serious mental illness from 1 health plan. Although DPC used multiple strategies to solicit participants, YES Health ultimately reached relatively few people, especially few young people. Study design (ie, recruitment of participants from only 9 randomly selected primary care practices) and human subjects considerations prevented DPC staff from having contact information for eligible individuals. Personal outreach to individuals might have attracted more members. Furthermore, One Care had gained considerable national attention,1,4,6,8 and its enrollees underwent extensive surveying, including 12 quality surveys administered by CMS and MassHealth.14 Anecdotal reports suggested “survey burnout,” or fatigue, among One Care members, even about answering questions from their peers with disability. YES Health participants were all aged 30 or older, which could have contributed to their preference for telephone rather than online communication. Finally, although DPC tried intensively to contact primary care practices, different approaches might have been more productive.
We involved 2 subgroups of One Care participants: persons with significant physical disability or serious mental illness. Although we recognized that members of these subgroups could prioritize different aspects of care when assessing their experiences, we chose to design a single survey (PDQ-S), which addresses shared concerns. Our cluster randomization did not treat these subgroups differently, and our study was not powered to consider differences between them. Nonetheless, in our discussion of results (see the “Results From Cluster Randomized Controlled Trial” section), we highlight some differences between these groups.
The “Study Results in Context” and “Implementation of Study Results” sections review important limitations of our study, including concerns about generalizability. We were not powered to examine results by individual practice or practice type (eg, practices operated by CCA clinicians vs contracted practices). The investigators were limited by our agreement with CCA from proactive outreach efforts with the PCPs serving One Care enrollees, largely because of the challenges and time commitment of implementing a new program involving complex, high-need patients. However, the disability advocates conducting YES Health—as representatives of One Care consumers—were free to approach these clinicians. We had only postal addresses, not email addresses, limiting our ability to communicate with the PCPs. Another limitation was our reliance on the rating category assigned by MassHealth to identify persons by disability type. Based on previous MassHealth claims, these rating category assignments were often erroneous.6
Two hypotheses or expectations underlying our research were that having empowered patients engage collaboratively with PCPs to inform quality improvement would be welcomed by physicians and result in better care quality. Because PCPs failed to engage with the DPC disability advocates conducting YES Health, we cannot assess whether the second hypothesis holds. We also do not know whether trying alternative means to create partnerships between health plan members and PCPs regarding quality improvement projects would generate productive dialogue. Future research should address both concerns.
Through YES Health, leaders in the Massachusetts disability advocacy community with significant physical disability or serious mental illness reached out via multifaceted efforts to engage their peers in evaluating the quality of their One Care services. Our project demonstrated the feasibility of individuals with disability designing and implementing this multipronged effort to gather data that were maximally meaningful to the One Care constituency. Nevertheless, over 12 months, YES Health attracted only 45 members—and their participation depended largely on $10 payments for each contribution (eg, answering the short, quarterly Survey Monkey questionnaire about care quality). Compared with going online, participants greatly preferred providing their input over the telephone by talking with YES Health staff, with whom they described a sense of ease and connection. Anecdotal reports about these conversations suggested that YES Health members found them meaningful.
Disappointingly, the cluster randomized controlled trial failed to demonstrate that YES Health—or reports to PCPs about baseline PDQ-S results—had any effect on improving care quality, as assessed through reports of One Care members with significant physical disability or serious mental illness. However, aggregate results from PDQ-S reveal that, while One Care has substantial achievements, more work is required to improve care for the particularly disadvantaged population it serves. For example, according to wave 2 PDQ-S results, only 41.3% of respondents feel their quality of life will improve in the future—not surprising given their substantial burden of chronic disease and functional impairments. Only 35.5% responded that their health was “better now” since enrolling in One Care. However, in wave 2, 51.0% said the quality of all the health care they had received was “better than before”; 59.1% reported having a care plan and 63.5% care teams, although One Care requires all members to have both care plans and teams. These findings suggest solid progress toward achieving this goal, but considerably more remains to be done.
Given the gaps between enrollees' perceptions of their care and the requirements of One Care, the failure of PCPs to respond to repeated efforts of the YES Health team represented the greatest lost opportunity of the study. Each of the 2-page, quarterly reports YES Health sent to PCPs contained recommendations about how identified quality problems could be remedied, in the view of these disability advocates. Whether a productive dialogue between One Care PCPs and the YES Health disability advocates could have improved care remains an unanswered question. Future research will need to assess whether direct collaborations between physicians and empowered patient representatives will improve care quality.
When individuals enroll in One Care, MassHealth assigns them a “rating category” based primarily on previous MassHealth claims. This rating category determines the capitated payment level. Rating category C2 indicates community-dwelling persons with “high behavioral health need”; C2B includes persons with comorbid substance use disorders; and C2A includes all other persons in this category. Rating Category C3 indicates community-dwelling persons with substantial needs for LTSS care: 2 or more ADL limitations and 3 skilled nursing days weekly; or 4 or more ADL limitations. C3B indicates persons with selected diagnoses associated with very high costs, and C3A indicates all other persons in this category. Our sample included persons in the C2 and C3 rating categories, as indicated in the CCA documentation.
People whose language preference was not known were included in the survey.
We also had resources for face-to-face, in-person administration, but no participant requested that option.
Applying the AAPOR RR4 formula to the wave 2 data, the numerator in the response rate formula is 426, and the denominator is 426 + eligibility rate × 284, where the eligibility rate is 426/436. This assumes that all nonrespondents have unknown eligibility (ie, it is possible that they could be dead or have a language other than English, Spanish, or American Sign Language).
For justification of using nonresponse adjustments to weights, see Chapter 8 in Lohr SL, Sampling: Design and Analysis. Brooks/Cole Publishing Company; 1999.
Percentages from PDQ-S results represent weighted percentages. We weighted percentages to ensure that the estimates provided here represent the entire underlying population, not just the sampled cases.
Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#IHS-1306-01424). Further information available at: https://www.pcori.org/research-results/2013/reporting-care-experiences-people-significant-physical-disability-or-serious
Iezzoni LI, Heaphy D, Donelan K. (2019). Reporting Care Experiences of People with Significant Physical Disability or Serious Mental Illness to Primary Care Clinics. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/2.2019.IHS.130601424
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
