In our results we have demonstrated that:
Despite increasing interest in and advocacy for co-production, there is a lack of rigorous evaluation of these approaches in acute health-care settings. In particular, there needs to be evaluation of clinical and service outcomes as well as cost-effectiveness of co-production approaches relative to other forms of quality improvement. The broader impacts on the values and behaviours of participants also merit evaluation (research question 1).
Completion of full and accelerated cycles of EBCD as a particular form of co-production in four stroke units was feasible. Both forms of the EBCD approach resulted in improvements in the quality of the stroke unit’s environment and increased activity opportunities for patients. Improvements were focused on issues that former patients and carers identified as part of the participatory EBCD approach. We found less evidence of positive change in enabling communication between staff and individual patients, which was more commonly task oriented than patient centred (research questions 2–4); these findings were evident in both the qualitative data and the responses to the PREMs in each site.
Qualitative evidence from different sources (interviews, ethnographic fieldwork) has shown that EBCD can lead to improvements in supervised and independent activity. This was achieved largely by increasing access to groups, both structured and staff-led groups and social groups facilitated by community volunteers, and by changing the use of existing ward spaces, which provided more opportunity for social interaction and informal group activity (research questions 2–4).
Changes in levels of social, cognitive and physical activity measured by behavioural mapping were inconsistent across sites and showed a mixed pattern of activity and inactivity in the small groups of patients observed pre or post implementation of co-designed changes (research questions 2–4). However, taking the broad measure of overall level of patient ‘activity or inactivity’, there was some improvement at sites 1 and 2 (full EBCD) but minimal changes at sites 3 and 4 (accelerated EBCD).
There were some signals in the post-EBCD cohorts’ responses to the PREM that at three out of the four sites there were more things for patients to do in their free time, and some indications that elements of patients’ experiences in the units had improved between the pre- and post-intervention data collection periods. However, these changes were not consistent across all sites and may reflect different patients completing the questionnaires at different time points rather than actual changes. PROM data indicated that the respondents were not atypical and had levels of physical impairment, dependency, emotional and social limitations congruent with national and international stroke statistics; however, rates of response to the PREM and PROM were low, varying from 12% to 38%.
Patients, family members and staff engaged well with both forms of EBCD and perceived that substantive changes had occurred. Co-design was a feasible method of stimulating new independent and therapeutic activities and prompted wider consideration of the influence of the stroke unit environment on patient activity in all sites (research questions 2 and 3).
Patients and carers played a significant role in highlighting priorities for improvement and sharing the work of co-design (research question 5).
There were no significant differences in experiences or outcomes between the full and the accelerated forms of EBCD. Accelerated EBCD contextualised to stroke units has the potential to spread across other stroke units and to other acute inpatient settings (research question 4).
Implementation of EBCD and of the co-designed changes was influenced by a number of factors and organisational processes, including established ward routines that were care and treatment focused rather than activity focused, and staff workload pressures. However, the structured and time-limited process of EBCD in both the full and the accelerated forms legitimised and supported participatory co-production activity. All participants recognised that increased activity needed to be embedded in everyday routines and work in stroke units. This aligned more easily with the existing working practices of therapists, but there were some indications that nursing work could also encompass prompting and supporting more social, cognitive and physical activity.
Communication between staff and patients which was patient-centred and supported activity was the most challenging to initiate and sustain (research question 6).
We now discuss how the CREATE findings compare with those from other research, and the strengths and weaknesses of the study. Finally, we discuss the implications for evaluating the implementation, spread and sustainability of EBCD and the overall impact on activity opportunities available for inpatient stroke patients.
Comparison with other research
Interventions to promote increased activity on stroke units
CREATE is similar to other studies that have set out to increase activity levels on stroke units, and we have shown mixed results. However, most previous research has focused on dose-driven interventions, including circuit class therapy and 7-day-a-week therapy.37 These have increased the amounts of therapy provided but have not increased meaningful patient activity outside therapy sessions. A more general approach to the environmental enrichment of stroke units with attention given to cognitive and social activity as well as physical activity is now gaining interest.2,87 The main studies,2,87 which have been conducted in Australia and have utilised controlled pre and post designs and are similar to CREATE, have evaluated the impact of a more stimulating environment on inpatient activity. The sample size was greater (n = 30) and the measures were focused largely on quantitative outcomes. Behaviour mapping results showed an increase in activity levels across all domains and sustained changes at 6 months post intervention. However, in these studies, the stimulating activities and changes made were driven by the perspectives of professionals, with no evidence of patient and carer involvement. The authors did highlight the benefit of using change management methods to address implementation, but, unlike in CREATE, they did not use a specified improvement approach.
Co-design approaches in acute health care
In accordance with the findings of our rapid evidence synthesis, we also found that co-design in acute health-care settings can be challenging and time-consuming to set up. Similar to other studies, engaging with co-design and implementing changes were viewed positively by patients, carers and staff. However, unlike many projects that have used EBCD, we incorporated more ‘designerly thinking’ into our project through the activities and support of our co-applicant Alastair Macdonald. We believe that this enabled greater creativity to emerge through the co-design groups and from the site champions.
We found no appreciable difference between the full EBCD cycle used at sites 1 and 2 and the accelerated form used at sites 3 and 4. This mirrors the finding of Locock et al.,60 although, unlike in that study, we developed our trigger films locally in stroke units instead of using a national database.
Strengths and limitations
Strengths
We believe that the strengths of CREATE lie in the design and the multiple data sources used in our evaluation and intervention (EBCD), and include a considerable range of data from interviews with stroke patients, families and staff (n = 155) and ethnographic field work (365 hours). This has enabled us to reach a deep level of understanding of the experience for patients, families and staff taking part in co-design, and the staff groups who were less directly involved in co-design, as well as the priorities and impact of the improvements on patient activity. We believe that our staged and iterative approach to qualitative analyses has facilitated an understanding of both the similarities and the discrete differences between sites, which have informed our recommendations about the use of EBCD as an improvement approach in other stroke units.
Our process evaluation also drew on multiple data sources, including researcher reflections, which were kept by all members of the research team (FJ, DC, KG and SH) throughout the project. This collaborative approach to sharing experiences, insights and analysis has provided a new level of insight into ‘the work’ of co-design in acute health care and what it takes to succeed and to sustain change.
We believe the cyclical approach used through EBCD enabled a close relationship to be gained with the research teams based in each site. Although this could be viewed as a limitation, the strategies used, such as encouraging teams to seek ways to overcome barriers, involve local community groups to support their work and celebrate their achievements, all helped staff, patients and families to gain a sense of momentum and change.
Limitations
Inability to use patient-level Sentinel Stroke National Audit Programme data
In our protocol, we stated that we would access routinely collected data at ward level to summarise and compare demographic data, age, gender and stroke severity (National Institutes of Health Stroke Scale and modified Rankin Scale) from a cohort of 30 patients pre/post implementation in each unit. However, gaining access to these data proved difficult and the time required to do so did not fit into our project timeline and would have exceeded the data collection period for sites 1 and 2.
We were able to access SSNAP Acute Organisational Audit data for sites 1 and 2, but the repeat of that audit was delayed and had not been conducted before we submitted our final report. We were unable to access organisational audit data for sites 3 and 4 or to add anything on patient-level data on stroke severity/dependency.
Behaviour mapping
As part of our mixed-methods approach we used behavioural mapping, a validated approach developed primarily to observe level and type of physical activity but more recently also used to record social, cognitive and physical activity for individual patients at 10-minute intervals over 8-hour periods. Although the method was feasible to use in acute stroke units, we believe that our results should be interpreted with the following provisos.
Of concern were the anomalies that arise when reporting behavioural mapping findings. Principally it can be possible for physical activity to be recorded as less across the observation period (epoch) even though the patient may be more socially active; in addition, if a patient is sleeping, the response to ‘no activity’ would be ‘yes’. Thus, ‘no activity’ needs to be interpreted with the number of patients sleeping in mind, as this influences the overall ‘no activity’ percentage. Behavioural mapping is better utilised in studies where the impact of discrete intervention is measured in the same cohort of patients over short time periods.28
Apart from the concerns about reporting outlined above, we believe that a number of other factors influenced the quality and relevance of behavioural mapping as a reporting method in CREATE. First, we consented patients the day before mapping took place, which led to restrictions in the numbers of possible participants. Numbers included ranged from 4 to 10, which meant that we were mapping the activity behaviour of only a small proportion of patients at any given time. Second, as our protocol determined, we did not include scheduled therapy sessions as part of recorded activity, and we also had a number of ‘unobserved’ recordings. For instance, if a patient was at an outside café or in the bathroom, they were not recorded as being active. Third, we believe that contextual issues such as staff shortages and the severity of disability of the inpatient caseload had a serious impact on the activity opportunities we could record. Nonetheless, the fact that we witnessed many instances of patients spending the majority of their time in bed or at the bedside with no interaction and nothing to do is a potent reminder that more work is needed to increase patient activity outside routine therapy in stroke units.
Low response rate to patient-reported outcome measures/patient-reported experience measures
We chose to use a validated PROM/PREM tool so that we could gain contextual data about the impact of stroke on patients post discharge and their experiences of being on a stroke unit. However, the return rate was low (< 40%) across sites, and in discussion with our NIHR manager and study steering group we used several strategies to increase this. We initially aimed to collect our pre-implementation PROM/PREM data retrospectively for 30 patients to coincide with SSNAP reporting periods (i.e. quarterly). We increased this to 6-monthly, which comprised two quarterly SSNAP periods, and we carried out repeat mailing and reminders. Overall, we believe that the post-implementation data collection period at sites 1 and 2, which was either side of Christmas, had a detrimental effect on the response rate, and, despite our efforts, we managed to reach our target of 30 returned questionnaires at site 2 only. Although the PROM/PREM has been validated for use with neurological inpatients, this is the first time, to our knowledge, that it has been used with stroke inpatients. In addition, the length of the questionnaire could have been a barrier and, overall, we believe that these surveys offered limited additional insight into the impact of CREATE beyond that gained through the semistructured interviews.
The challenge of the critical distance of the research team
We introduced a number of stages in our protocol to try to reduce researcher bias, such as using a standardised protocol and behavioural mapping tool for recording patient activity episodes and ensuring that recording sessions were spread across a 10- to 14-day period to allow maximum opportunity to account for variations in activity at different times of the day and on different days of the week/at weekends and the presence or absence of family members/visitors. Our behavioural mapping protocol also excluded individual therapy sessions to reduce the likelihood that increased activity levels would be captured later in the inpatient stay when individuals would be more active in therapy.
We were unable to use a purposive sampling approach as recruitment for behavioural mapping proved challenging at all four sites and, thus, given the convenience samples we worked with, we could not select patients at different points in their inpatient stay or with different levels of stroke severity.
However, our behavioural mapping results were clearly variable between the pre- and post-EBCD activity periods, indicating that, although patients, relatives and staff may have been more aware of the EBCD project and although opportunities for activity may have increased, this was not necessarily reflected in individual behavioural mapping profiles, which tended to indicate low levels of individually focused activity.
We were unsure whether there were any changes in staff behaviour as a result of researchers conducting behavioural mapping and observations/interviews. Our field notes and memos show that, across all sites, some early observations raised staff members’ awareness of the project and may have influenced some of their interactions with stroke survivors but, as in most observational studies, this effect quickly dissipated. As observations (pre and post) were extended over 10 sessions over 2–3 weeks and interspersed with behavioural mapping recording, staff became used to the researchers’ presence and we observed very little evidence of change in staff behaviour in favour of increased patient activity outside therapy during these (pre or post) periods of observation.
We do acknowledge the potential for the research fellows who were involved in supporting EBCD and facilitating co-design groups to develop a sense of ownership and investment in the project at each site but would point to the actual time spent at the sites being episodic, which meant that there was separation from the day-to-day work on the units. Overall, as a research team we had different roles and different levels of involvement. The research leads (FJ and DC) had considerably less time in the sites and less involvement in the co-design groups; however, they did conduct (pre and post) observations and interviews and added a different perspective on observed and reported activity from those of the researchers who routinely facilitated EBCD activities.
At sites 1 and 2, quite long periods of time separated the pre and post observations and interviews, and we sought different as well as similar interviewees (i.e. those directly involved and those not involved with EBCD). The process was broadly similar at sites 3 and 4 but, although the overall time was reduced, a gap of around 6–7 months remained. As a team, we were mindful of the need for objectivity in reporting on what was observed.
In our analysis meetings and in our integration of the data for the process evaluation, we had an opportunity to review field notes, summary memos and researcher reflections, which helped us identify factors that may have influenced the researchers’ perceptions and reporting of activity (or absence of activity) at sites; this included Study Steering Committee members robustly reviewing our presentation of the emerging and final data.
However, as stated in Chapter 5, one of the main challenges the researchers faced was that their role included both data collection and the co-ordination of EBCD and co-design meetings. Although there were examples of autonomous and proactive behaviours by staff, patients and family members, an unintended consequence of the researchers’ regular presence and willingness to facilitate improvement could have been a reduced commitment among staff to collective action and undertaking the work required to actually implement changes to practices. Paradoxically, the absence of researchers in future projects might facilitate greater commitment and engagement from clinical teams, notwithstanding the need for champions to emerge in each site to drive forward the improvement cycle.
Implications for health care
The added value of using co-design to initiate change
Traditional approaches to improving activity on stroke units have focused largely on setting national targets for therapy intensity. These have failed to have an impact on the stroke unit environment and the range and quality of activity opportunities outside formal therapy provision. We now question the narrow focus of this ‘top-down’ approach to therapy and its lack of consideration of the broader rehabilitation concern of increasing activity. Outside the narrow focus of clinical audit, action is required at both national (guideline recommendations) and local levels to increase therapeutic activity outside therapy. This needs to be based on a refocusing on rehabilitation post stroke, needs to be addressed at stroke unit team level (i.e. taking a multidisciplinary approach), and will benefit from the use of participatory service improvement methods such as EBCD.
The success of participatory approaches and EBCD used for the first time in acute stroke services lies in the knowledge that patients, families and staff decided and agreed on what mattered most to them as well as what could be achieved. We believe that the strength of EBCD in both its full and its accelerated form is the facilitated, structured, participatory and time-limited process. The nature of the ‘work’ in CREATE was fundamentally different from usual staff or externally driven quality improvement initiatives in stroke, and it prioritised the participation by stroke survivors and their families in more creative, tactile and relational interactions and outputs to improve opportunities for independent and supervised activity.
It was also evident that CREATE provided distinctly different experiences for staff who engaged in EBCD and that the changes agreed were driven not by external policy pressures or by local organisational demands, and not by external audit, but rather they were shaped very clearly by the experiences of patients and carers whom staff had recently cared for or treated, and were locally determined by small groups who took responsibility for bringing change about. This motivated groups to work together in co-design meetings to define ways to implement the new working practices, what Wenger refers to as ‘communities of practice’ in each site who were central to bringing about change.88 The involvement of patients and carers increased the accountability of the staff who participated and made it more likely that the planned changes would proceed. Co-design also facilitated carers’ and volunteers’ continued involvement in activities and directly contributed to the sustainability of the changes made to the day-to-day working of these stroke units.
Co-design activities in CREATE also initiated new and ongoing engagement with local people and/or organisations for whom the hospital is a key part of their community. The positive aspect of greater community involvement was notable in one of the London sites threatened with closure at various times. As a result of increased engagement with art communities, CREATE was viewed locally as a ‘good news’ story in terms of providing examples of how co-produced changes improved inpatient services in the ‘local’ hospital. shows a mayoral visit that celebrated the opening of the new post-EBCD ward at site 1. We believe that acknowledgement and celebration of the efforts made by staff, patients and carers – the final part of the cycle of EBCD – is an important factor in raising awareness of change and sustaining achievements.
Mayoral visit to celebrate changes to ward at site 1. Reproduced from the Sutton & Croydon Guardian with permission from Epsom and St Helier University Hospitals NHS Trust.
The ongoing challenge of (in)activity in stroke units
Culture change in any organisation is challenging, and in our project this was no different. The process evaluation highlighted the importance of organisational buy-in but also the challenges of achieving this at unit level and beyond. It was also clear that EBCD provided both a structure and a space for changes aimed at directly improving the experiences of stroke patients and their family carers. As a result of making these changes, the morale and commitment of staff also seemed to improve. Nonetheless, although improvements in activity opportunities were observed, the process evaluation also highlighted the challenge of involving and keeping engaged large groups of staff in the participatory change process. Similarly, differences in staff members’ perceptions of their role in rehabilitation and enacting this in day-to-day working practices highlighted that interdisciplinary rehabilitation and increased activity promotion across a whole stroke team would require more sustained intervention targeted at how teams work in this context, which was beyond the scope of CREATE. Although the tangible improvements to the space and activity opportunities was visible across all stroke units and validated by our qualitative findings, when behaviour mapping was used as an objective measurement tool no consistent change was found in the proportion of time on weekdays and weekend days that stroke patients in the participating units spent on physical, social and cognitive activities. The effectiveness and long-term impact of lengthy periods of co-design work (ranging from 4 to 9 months in this project) on the independent and supervised activity of stroke patients is still uncertain. Currently, work routines in participating units and day-to-day interactions between staff and patients are largely task driven, focusing on care needs and delivering scheduled therapy. Interactions that facilitate greater social exchange, cognitive activity or physical exercise are relatively uncommon, and we believe that to increase such activity opportunity requires a fundamentally different approach to how therapy and nursing contribution to rehabilitation is viewed and delivered so that stroke unit environments and work practices become more conducive to activity and so that activity is seen as a priority and part of the rehabilitation work of all staff, not only therapists.
At all CREATE sites we found concern that ‘something must be done’ and a willingness for staff, patients and families to work together to make improvements. We believe that, for this approach to be used across other stroke units, local facilitation needs to be provided by a member of staff who has protected time to be allocated to this work. Change required buy-in and commitment from multiple stakeholders including senior management to validate the shift from achieving national performance targets to a cultural recognition of the therapeutic value of a stroke unit. Change was facilitated when stroke unit staff worked with volunteers and people from the community. This will take time, but trigger films such as those developed in this project can help gain traction to make changes, some of which can be initiated quickly and relatively cheaply (e.g. photo-hangers by bedsides, and the ‘a little something about me’ board).
Costs and sustainability
It was not within the remit of CREATE to evaluate the cost of EBCD in its full or accelerated form, but we found no appreciable difference in the impact or extent of changes between the full and the accelerated EBCD cycles. We therefore conclude that accelerated EBCD is a feasible and efficient improvement method in stroke units and other acute care settings. However, we believe, based on our findings, that to streamline the processes further would be a risk to its success. Each of our sites has highlighted inherent contributors to effecting change in such a complex clinical environment, including having sufficient time to build a community of practice through the co-design work. Time is critical to the development of a sense of responsibility to the process (i.e. to deliver what was agreed), and to plan for and enact change, which often involved working with others to navigate bureaucracy and required the creativity and resourcefulness of co-design team members. This was seen at site 1, when the delay led a general manager and the head of nursing to expedite changes and seek solutions to funding issues, such as decorating the bays themselves. At site 3, the stroke team’s fundraising efforts were recognised by the trust chief executive, who agreed to match the amount raised. Finally, a stroke physician at site 4, frustrated by delays, brought about a solution to the lack of iPad stands by repurposing research and development funds. We believe that these actions would not have been possible within a shorter time frame.
Sustainability is also difficult to measure, and this too was influenced by operational and structural changes outside our control, such as hospital redesigns and staff shortages. Post completion, each site continued to make use of the activity improvements and some added further to these. For example, at site 1 the collaboration with local artists continued and a number of activities have been delivered for patients at weekends, culminating in a new mural and artwork in the ward space. At site 2, staff meetings that used to take pace in the reclaimed day room now take place in a shared therapy office between the rehabilitation room and the acute ward. At site 3, the new shared kitchen facility has been completed and this space is now accessed by staff, patients and families. At site 4, the day room previously used only for wheelchair storage has been maintained as a new activity space. The wheelchairs were moved to a room on the floor above and staff have to factor collecting and returning these wheelchairs into their working day; however, there have been no reports of this process eroding therapy time or inconveniencing staff. In addition, at site 4 there is continued use of the open window area that was previously used mainly for staff breaks, equipment storage and display of Stroke Association materials. Equipment is now stored in other areas close to the ward nurses’ station, and staff breaks are taken off ward or in the ward offices.
Implications for research
The cost-effectiveness of the methods used in CREATE are unknown, although quality improvement methods, such as the accelerated EBCD used at sites 3 and 4 could be highly cost-effective if improvements can reduce the inactivity of inpatient stroke patients, contribute to an increase in independence in activities of daily living and reduce length of stay. Equally, the participatory approaches used in EBCD can have a positive impact on the morale, meaning and purpose of staff in the face of increasing staff shortages and caseload pressures. Staff in our study felt that there was little time for any creative thought and relational work in their day-to-day practice, and the opportunity to take part in CREATE, to make a difference and to do something positive, was viewed as constructive across all stroke units.
We believe that CREATE has added to the knowledge about co-design in acute health care and that our methods could be applied and evaluated across other stroke units and other acute health-care environments. Our findings support Greenhalgh et al.’s key principles for achieving impact in co-design, including the need ‘to frame this research as a creative enterprise with human experience at its core’80 and an emphasis on process, such as the nature of relationships, leadership style, governance and facilitation arrangements. Our rapid evidence synthesis highlighted common barriers encountered in co-production approaches and these barriers were similar in this study, which was similarly thwarted at times by a lack of resources or managerial authority to bring changes.16 However, unlike other research projects, we had little difficulty recruiting patients and carers and retaining them in the project. This, we believe, was largely a result of the dedicated efforts of our local researchers and the willingness of staff in the co-design groups to both engage with and support stroke survivors and their families to participate in the EBCD process.
Any further research in this field requires early engagement and commitment from estates, general management, senior executives, and communications at the start of the project and to utilise methods to retain interest throughout and after completion. The stakeholder mapping exercise at the start of EBCD was critical.
Early consideration of community/voluntary sector engagement is also important and is a key learning point from sites 1 and 2, which used the full EBCD cycle. This enabled awareness-raising among and interest from a range of local community groups, which subsequently added to activity opportunities through art, exercise groups, reading and music.
Finally, several research questions have emerged from our study, which are as yet unexplored:
What are the additional steps that would be required to change the culture of activity on a stroke unit? In particular, what changes are required to achieve greater consistency in the use of enabling communication by all staff?
Can the CREATE accelerated EBCD approach using stroke-specific trigger films be used across other acute stroke units, and what would be the contextual adaptations required to enable similar levels of improvement?
What are the ‘costs’ of setting up accelerated EBCD in terms of time and people? How sustainable is the approach in a stroke unit and how frequently should an EBCD cycle run?
Are there transferable lessons outside a stroke unit environment (e.g. does it matter whether the ward is ‘specialist’, or for short, medium or long stay)?
What additional tools and methods are required to provide a more effective way of measuring any changes in activity, given our concerns about the sensitivity of behavioural mapping?
What additional approaches would be required to change the culture of activity on a stroke unit; how can ‘enabling activity’ be viewed as the work of all staff, including nursing staff?
To what extent can patients/families and local communities support sustained activity, providing greater opportunities for social, cognitive and physical activity outside scheduled therapy provision?
What is the longevity of the changes made, and will the CREATE stroke units continue to work with patient and carer groups to review these and make improvements?
What aspects of EBCD promote lasting change?
