Training attendance

Data sample: attendance records were recorded by research midwives for all training sessions they delivered.

To analyse training attendance, we descriptively summarised the number of midwives who attended training from each community midwife team in the intervention cluster [e.g. 89 of 92 (97%) of midwives from n = xx community teams].

Training observation

One of two members of the research team joined and silently observed four of the main training sessions. The researchers checked whether key statements were articulated, which were listed in bold on the midwives’ training presentation slide notes. The key statements were ticked off once delivered, permitting quantitative monitoring of training fidelity. The researcher also qualitatively described the spirit of delivery as intended by the protocol.

The quantitative data constituted the number of key statements delivered – this was summarised descriptively via the proportion (%) of key statements delivered, as well as any key statements that were consistently omitted by the facilitators.

The qualitative data gathered from this source were evaluated via content analysis⁵¹ to assess: (1) what went well, (2) whether any aspects of the mode of delivery needed improvement and (3) what barriers could pose a challenge to training implementation. The data were coded deductively, as well as checked inductively, to establish the main themes that arose from that data set.

Both analyses were synthesised narratively, highlighting any points for training refinement.

Pre-/post-training confidence questionnaires

The questionnaires were completed by consenting intervention midwives before and after the training sessions. There were eight questions that were a part of the survey, scored using a 5-point Likert scale (0 = not at all confident to 4 = completely confident), giving a range from 0 to 34. Therefore, a higher overall score implies a greater confidence in discussing and teaching PFME in antenatal care. The data collected from this source allowed for individual question quantitative summaries.

A summary score was produced for each question for both the pre- and post-training questionnaire. The question summary scores were summarised with descriptive statistics (e.g. median, IQR, mean, SD). Box charts and bar charts were used to visualise these findings. The pre- and post-training confidence scores were used to calculate any change in confidence before and after training, per question. The appropriate parametric or non-parametric repeated measures inferential statistic was used, with p < 0.05 indicating a significant change in confidence.

Training evaluation questionnaire

An anonymous questionnaire was completed post training by intervention midwives. The questionnaire involved six 10-point Likert scale questions (0 = negative response, through to 10 = very positive response, to the training) as well as free-text responses about the acceptability of the training content and delivery. The data from this questionnaire allowed quantitative analyses for each question, but the six Likert scale questions were not designed to be summed; the free-text data were analysed qualitatively.

Each Likert scale question was summarised descriptively (e.g. median, IQR, mean, SD). The free-text responses were analysed using content analysis⁵¹ for each question individually and then administered on the data set as a whole. It was initially an inductive analysis, using a subset of questionnaires to develop categories and a coding framework for use with the remaining questionnaires. Text was analysed for the appearance and frequency of specific content, and the process followed the four steps outlined below.

1. Inductive analysis to identify codes. Coding focused on capturing manifest content (i.e. content which was visible and obvious rather than implicit). This is the most appropriate level of analysis where brief, possibly single-word, responses were recorded. Codes were grouped into categories for each question.
2. The first round of analysis grouped codes into categories for each question to create a coding framework, using a subset of questionnaires.
3. The coding framework was applied to the remaining questionnaires and the frequency of occurrence of coding categories was recorded.
4. A second round of analysis was conducted to look at the data set as a whole (rather than by question). Data were analysed deductively based on the BCW, and inductively to identify any additional themes arising within the data.

Quantitative and qualitative analyses were synthesised narratively, highlighting any points for training refinement.

Interviews were conducted with intervention midwives and champions (post training and end of study), and control midwives (end of study) and women (end of study, post questionnaire completion).

Thirteen interviews, each for 30–60 minutes, were conducted with intervention midwives and champions post training. They aimed to gather information on the midwives’ experience of the training and of implementing the training into their practice, including any limitations they experienced and suggestions to improve the training. The interviews also focused on midwives’ views on the acceptability of the training session and its content. Interview topic guides were informed by the logic model (see Appendix 2, Figure 4), BCW theoretical framework and Sekhon’s acceptability framework.⁴⁹

Data were analysed using a hybrid thematic approach, combining deductive and inductive analyses.⁵² Deductive (or theoretical) thematic analysis was driven by the logic model for the intervention (see Appendix 2, Figure 4), the BCW theoretical framework that was used to guide intervention development,³⁶ and Sekhon’s acceptability framework,⁴⁹ which helped inform the topic guides. Inductive thematic analysis enabled identification and analysis of any novel themes that did not fit into the predefined theoretical framework.¹¹ Telephone interviews were transcribed by an approved transcription service. Online interviews conducted via Zoom were auto transcribed and checked for accuracy.

The phases of thematic analysis were as follows:

1. checking and reading of transcripts and data familiarisation
2. initial coding using deductive and inductive approaches
3. 10% of transcripts will be coded by a second analyst
4. identification of themes
5. reviewing of themes
6. theme definition and naming
7. finalising and reporting of analysis.

Analysis was discussed by analysts and research team members to support the interpretation of the data.

Champion monitoring

Data source: champions recorded their activities related to this role using a template form provided in the champions’ manual. Entries included, for example, the number and nature of requests for support from the midwives in their team (number of requests for advice or onward referral) plus any open-text data reflective journal detailing the support they offered to midwives, any observations regarding any possible contamination between intervention and control teams, as well as their general experiences of being a champion.

Quantitative data were summarised descriptively. Free-text and qualitative data were summarised using content analysis (as detailed previously).

Distribution of resource bags for women

Data source: the trial research team recorded the number of resource bags given to teams. These data were summarised descriptively.

Fidelity of intervention implementation questionnaire

An anonymous questionnaire was completed during the implementation period by 59 intervention midwives. The questionnaire involved seven 5-point scale questions about implementing APPEAL training to the women in their care (4 = ‘yes, all women’ through to 0 = ‘no women’); one 4-point scale question about appointments (3 = ‘at all appointments’ through to 0 = ‘never’); one ‘yes/no’ question about referrals made (including frequency of referral if yes); two questions, one on barriers and one on facilitators to implementation, each with nine categorical response options (tick as many that apply) and a twelfth question inviting any other comments (free-text responses). The data from this questionnaire allowed quantitative analyses for each question, but the questions were not designed to be summed; the free-text data were analysed qualitatively using content analysis as detailed previously.

Quantitative and qualitative analyses were synthesised narratively, highlighting any points for training refinement.

Women’s questionnaires

Sections 3–5 of the APPEAL outcomes questionnaire for women were further analysed within the process evaluation in conjunction with, and after, the preliminary statistical analysis had taken place. Summaries of intervention and control arm responses to sections 3–5 (PFME performance, information received from midwife, PFME self-efficacy and adherence) were undertaken as part of the main trial analyses and then incorporated into this process evaluation.

Responses to section 3 supported identification of midwife fidelity to the PFME intervention for women and possible contamination between clusters (see Table 18); section 4 would indicate women’s self-efficacy for PFME; and section 5 would indicate women’s adherence to PFME. Response rates to these sections indicated the feasibility and possible acceptability of collecting PFME outcome data from women.

TABLE 18

Identifying fidelity to PFME intervention – interpretation of responses to women’s questionnaire section 3

Responses to questions in section 3 were reported narratively using descriptive summaries (e.g. range with mean and SD or mode and median) where appropriate.

Section 4 contained the 17-item Chen PFMESES.⁴² Items are scored using a 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree), with a possible summed score range from 17 to 85 (higher score indicating greater self-efficacy for performing PFME). The scale contains two factors: (1) belief in PFME execution and benefits (items 1–4, 10, 12–17); and (2) belief in performing PFME as scheduled and despite barriers (items 5–9, 11). Missing data were examined and reported, and any imputation undertaken in accordance with the main trial analysis. A total self-efficacy score and subscores for each factor were summarised for the intervention and control arms. (NB as described earlier, the omission by BCTU of four questions rendered this full scale invalid.)

Section 5 contained the six-item EARS.⁴³ Items are scored using a 5-point Likert scale (from 0 = completely agree to 4 = completely disagree) with a possible summed score range from 0 to 24. Items 1 and 5 are reverse scored. A higher overall adherence score indicated better adherence to exercise. Missing data were examined and reported, and any imputation undertaken prior to statistical analysis in accordance with the main trial analysis.

Associations between variables that may explain differences in PFME practice, adherence and self-efficacy were explored descriptively. For example, for women who performed regular PFME: (1) what proportion were told about PFME by their midwife? (including how often they were told to perform PFME), and (2) what proportion were given a resource bag? Did women who reported being told about PFME by their midwife and/or who were given a resource bag, also report greater adherence and/or self-efficacy for PFME? These data were summarised descriptively.