INTRODUCTION

In 2018, the College of Midwives of Ontario (CMO), the provincial regulatory body, expanded the scope of practice to allow midwives with the appropriate training to perform ultrasound scans to improve pregnancy care.¹ Following this, a needs assessment of Ontario midwives highlighted their desire to learn and apply sonography skills, specifically, point of care ultrasound (POCUS), due to the convenience of this technology. Its versatility is due to the ability to aid in assessing various clinical issues such as confirming viability, establishing gestational date, and evaluating fetal well-being.² Despite the CMO’s regulatory change and a strong desire to learn sonography, sonography has not traditionally been part of the pre-qualification curriculum for midwives. There also have not been any dedicated post-qualification courses that address the role and scope of sonography specific to midwifery.

Currently, there is minimal literature examining the inclusion of POCUS in Canadian midwifery; however, globally, evidence suggests that midwives successfully use bedside sonography to inform clinical decisions that may improve perinatal health outcomes. Examples include assessment and management of vaginal bleeding or confirming amnionic fluid levels.^3,4 Although international midwifery education core competencies include minimal sonography content, it has been demonstrated that following additional POCUS training, midwives can interpret sonographic images with over 99% accuracy.⁵ Midwives reported using POCUS for confirming intrauterine pregnancies, estimating gestational age, and identifying fetal presentation and placental location.⁶ This research also found no significant difference between midwives’ immediate and one-year post-course objective structured clinical examination (OSCE) scores, illustrating their long-term skills and knowledge retention of this learning.⁶

To provide the appropriate training for midwives in Ontario, a McMaster-Mohawk POCUS Course for Primary Maternity Care was designed to provide entry-level POCUS skills — the first course available for midwives and other low-risk obstetric care providers. The course was designed to provide the knowledge, skills, and judgment to perform ultrasounds during the midwifery course of clinical care.⁷ To facilitate POCUS learning, this course used online modules, a two-day hands-on workshop, and a clinical practicum. Online module topics included the physics of ultrasound and core elements of scans throughout all trimesters of pregnancy. The workshop allowed learners to practice the clinical skills learned in the modules with pregnant volunteers and computer simulations across all three trimesters, with real-time feedback from sonographers. The course offered a student-to-instructor 3:1 ratio over the two days. After the workshop, learners were encouraged to continue to scan as frequently as possible, even if not supervised by a preceptor. This was so they could continue to develop their skills and remain familiar with the techniques acquired. Learners were also responsible for organizing a community practicum to obtain additional scanning time, underpinned by a competency assessment framework and under the guidance of a preceptor.⁷

Learner assessments of POCUS competency were done at three-time points in this course. First, learners were required to complete module-specific quizzes before attending the two-day workshop. Next, clinical skills were evaluated in the workshop through a series of simulated scans (OSCE). Finally, skills were further assessed in the practicum using a logbook of second and third-trimester scanning competencies, which were evaluated and signed off by a sonography preceptor when successfully achieved.⁷

Completing the course results in a micro-credential in Point of Care Ultrasound. A letter of completion for Level 1 POCUS is provided after the online modules and hands-on workshop are complete, and a final letter of completion is issued to the learner for Level 2 POCUS after the practicum is completed.

The first iteration, of course, occurred in 2019, and our aim for the current study was to evaluate this educational intervention to understand the educational impact of this new sonography training. The Kirkpatrick framework for educational outcomes underpinned the evaluation because of its clear steps in robustly evaluating an educational initiative. The Kirkpatrick model is a four-level pyramid framework commonly used to assess the various effects of education interventions.⁸ The model measures reaction, learning, behavioral change, and impact (Figure 1). “Reaction” refers to participants’ perceptions and experiences of the training, while “learning” reflects acquiring knowledge and skills.⁸ “Behavior” and “impact” focus on deeper learning as demonstrated through behavior change and application, as well as integration into the organization of client care (Figure 1).⁸

Figure 1. The Kirkpatrick Model

METHODS

The McMaster-Mohawk POCUS course was evaluated using a concurrent triangulated design. The data collection tools included four surveys (pre-course, post-course, four-months post-course, and one-year post-course) and semi-structured interviews. As POCUS in midwifery is a reasonably unexplored topic, triangulation allowed for multiple instruments to collect data over one year from October 2019–2020, with the data analyzed sequentially and then combined to generate patterns and themes.^9,10 The surveys developed and tested by our research team comprised five- and seven-point Likert Scale and open-ended questions. While not formally validated, the surveys were pilot-tested by peers, midwives, and researchers for face and content validity. The semi-structured interview questions, also developed by our team, allowed participants to expand on their survey responses and share their opinions in more depth. Quantitative results were analyzed in R and Excel using descriptive statistics to understand the general data patterns.^11–14 Qualitative results were analyzed using a Grounded Theory approach through a combination of Corbin, Strauss, and Charmaz’s approaches to coding.^15–19 Upon completion of selective coding, the emerging themes were reviewed about the four levels of the Kirkpatrick model to evaluate the educational value of this new curriculum.⁸ Steps taken to ensure the trustworthiness of the results included member checking, use of a second coder and peer debriefing with the research team.

RESULTS

The initial offering of the McMaster-Mohawk POCUS Course occurred in 2019. The course was advertised through McMaster Continuing Health Science Education and was open to anyone providing low-risk obstetric care. The online modules were launched in September 2019, and the two-day workshop occurred in late October 2019. All 18 learners enrolled in the course were invited to participate in this study, and all (17 midwives and one family physician) consented. These participants were from across Ontario: 83% from Southern Ontario and 17% from Northern Ontario.

For survey response rates, 100% of participants completed the pre-course and immediate post-course surveys. The four-month post-course (February 2020) and one-year post-course (October 2020) surveys had 61% and 56% response rates, respectively. The interviews took place in Spring 2020 with five participants. Response rates decreased throughout the study, likely due to the increased time passed since the start of the course and the struggles healthcare providers experienced in the Coronavirus-19 (COVID-19) pandemic.

We present our findings below according to the Kirkpatrick levels (reaction, learning, behavior, and impact) to show the multiple layers of this program evaluation.

Reaction

Participants’ reactions to the curriculum centered around exposure to unfamiliar content. They reported struggling with some of the material, particularly the physics content because it was very different from previous midwifery training. They expressed that the course required more time than expected to feel prepared for the workshop. Some participants also perceived that hands-on learning would be essential to consolidate the content in the online modules. One participant shared: “I’m not familiar with using that type of technology. So, it was really abstract to me, to be introduced to this for such a short time period when my skill training is definitely more hands-on experience where I just found the technology confusing.”

Participants were pleased with how well the content aligned with their learning needs, and they expressed interest in more education to expand their POCUS knowledge and skills. They perceived it to be very relevant to midwifery. Immediately post-course, 82% of participants were interested in learning about POCUS. These sentiments decreased to 73% and 70% in four-month and one-year post-course surveys. Participants felt that a refresher course would benefit those with extended time between the workshop and the practicum to ensure they remembered the module and workshop content before starting their placement.

Learning

Participants completed five online modules to prepare for the workshop. They mentioned how the modules introduced them to the didactic knowledge of core concepts. However, the time learners spent with sonography experts during the hands-on course was foundational to developing their knowledge and skills. As expressed by one participant, the course valued “being able to work with really experienced sonographers that were extremely helpful and really good at guiding me when I had no idea what I was doing.” Close contact with instructors ensured questions were answered and guidance with sonography skills was provided.

Unfortunately, many participants struggled to secure a clinical practicum following the workshop. Ultrasound clinics could not take on midwifery learners besides their other students, particularly with the uncertainties of the COVID-19 pandemic. Despite continued effort and interest, only two participants could complete the practicum within the study period. However, those who did complete the practicum commented on how important it was to work with their preceptor to solidify their POCUS knowledge and skills. The additional practice scanning time in the practicum made them faster and more accurate. One participant summarized the significance of the clinical practicum: “if I didn’t do the placement, I don’t know how comfortable I would be.”

Those unable to organize a practicum were encouraged to continue practicing scanning, even if not under the supervision of a preceptor. However, many of them lacked access to an ultrasound machine, and they expressed feeling that they were not able to progress their skills beyond the module and workshop learnings.

Because I’ve had no access to more training, obviously, the knowledge that I had or learned over the course has faded. I have the materials that I would review if I would get access to clinical training but at this point, [I am not] more comfort with POCUS than… prior to the training.

As a result of a lack of opportunity to practice outside of the workshop, participants’ confidence in their skills decreased from 69% post-course to 44% at the end of the one-year follow-up. Concerning how comfortable and competent participants felt with their POCUS skills and knowledge, only 11% were generally satisfied with the technology after completing the online modules; however, this grew to and remained at 50% after the workshop and the one-year follow-up, respectively (Figure 2). Participants reported feeling most comfortable performing scans for viability in the first and second trimesters (up to 89% and 78%, respectively). One hundred percent of participants reported feeling competent to perform a scan to confirm fetal presentation immediately post-course (Figure 3).

Figure 2. Participants’ POCUS Comfort Level Over Time

Figure 3. POCUS Comfort Level by Indication Throughout the Study

Another barrier reported by participants that impacted their learning included the financial and personal costs of enrolling in the course. Participants came from across Ontario. Many were responsible for significant travel costs in addition to course tuition. Organizing off-call time and coverage at their practice was an additional barrier to taking the course or completing the practicum.

Behavior and Impacts

At the end of the study, despite the limited opportunities for practicum placements, 60% of respondents from the one-year post-course survey shared that they were applying POCUS monthly. Being able to use POCUS as part of their clinical assessments was seen by participants as helpful for improving clinical efficiency. They could offer results and a care plan promptly. Participants also described how they could provide more personalized care, relieving clients’ anxieties by showing them the ultrasound images in real-time. The indications that participants felt most likely to use point-of-care scanning in their clinical practice included fetal presentation (100%) and viability of the first and second trimesters (89%). Participants were less inclined to use POCUS for ectopic pregnancy, biophysical profiles (BPP), and assessment of placental location.

DISCUSSION

Our results shed light on the reactions, learning, behavior change, and impact on practice for midwives in Ontario regarding point-of-care ultrasound. The participants in our study expressed excitement about gaining competency in using ultrasound during clinical practice.^20,21 Personal interest in learning will continue to be a significant influence on how midwives engage in continuing education. Participants also commented on how well the curriculum aligned with their clinical needs. The curriculum design was guided by an Ontario midwives’ needs assessment, explicitly asking what participants would be interested in learning and applying POCUS.² This step was critical to ensure the curriculum was relevant, and future course developers are advised to replicate this process.

Learning sonography skills was complex; however, participants benefitted from extensive practice scanning time throughout the course. The low ratio of students to instructors (3 to 1) at the hands-on workshop ensured that participants could ask more questions and that instructors could closely watch their scanning techniques to provide feedback; therefore, small class environments should continue in future POCUS courses. The limited number of participants who were able to complete the clinical practicum illustrates a barrier to learning that needs to be addressed in future offerings of this course. Possible solutions include extending the hands-on workshop to allow for more time to gain competencies, developing a roster of sonographers available to act as preceptors, and exploring the role of funding for these preceptors as a retention strategy.

The higher levels of the Kirkpatrick framework behavior and impact —were demonstrated in our findings when participants described how beneficial point-of-care scans were to evaluate fetal presentation or viability of the first and second trimesters as part of their clinical care. Participants in our study had completed the Level 1 POCUS micro-credential, and they were encouraged to continue scanning on their own if they had access to a device, even if they could not formally organize a practicum. Our finding that many of these participants continued to use these skills in practice speaks to the clinical relevance of POCUS for improving clinical care.

The application of this learning during clinical decision-making and the potential impact on clinical practice is likely influenced by the volume of scanning opportunities.²² This was reflected in the experiences of the few participants who completed the practicum when they described the importance of this time to refine their sonography skills. However, it is salient that determining a sufficient number of scans or scanning hours to assess and maintain competency for point-of-care ultrasound remains controversial among providers in disciplines outside of pregnancy, such as emergency departments, nursing, and emergency medical services.²³ Despite the importance of the practicum as demonstrated in our study, at present, there is no regulatory standard regarding the competencies required for midwives to use POCUS; instead, they work toward ensuring they have the knowledge, skill, and judgment to perform the skill safely. Further research and discussion about training, assessment, and maintenance of competency for midwives is warranted.

Our study has a few limitations. One limitation was that to protect participants’ privacy, we did not collect identifiers in the surveys. This prevented us from being able to compare individual measures of learning over time across the four surveys. Our research also relied solely on participants’ self-assessment of their learning via survey and interview responses. Initially, our study design included participants repeating the same OSCE assessment in the Spring of 2020 and comparing these scores to the Fall of 2019 to assess their skill maintenance objectively. However, the onset of COVID-19 meant that an OSCE could not be run safely in person, and this plan was eliminated. Finally, the lower response rates to the surveys at four months and one-year post course may have introduced some sampling bias; given potential differences between respondents and non-respondents, our ability to conclude the entire participant group based on the two later surveys is limited.

CONCLUSION

The first iteration of the McMaster-Mohawk POCUS Course was a novel opportunity to introduce midwives to obstetrical sonography. While learners perceived the content to be challenging, midwives were excited and engaged in the content because they wanted to provide more comprehensive obstetrical care. Participants appreciated the alignment of content with their clinical needs and the extensive practice time with pregnant volunteers in a safe learning environment during the hands-on workshop. Unfortunately, minimal learners were able to complete the clinical practicum. The practicum was perceived as highly beneficial for refining skills and improving comfort and confidence levels. Strategies to address the barriers to achieving the practicum must be addressed for future cohorts.