 Well, I am really pleased to have the privilege to introduce Kate Webber. She's a certified nurse midwife in Atlanta, Georgia, USA, and she's been practicing since 1999 and teaching at Emory's Nell Hodgson Woodruff School of Nursing since 2004. Having completed a dissertation focused on midwifery education and workforce, she's interested in working to build the midwifery workforce, not only to address provider shortages and maldistributions, but also to make the midwifery model of care a standard available to all women. She has attended over 1,200 births and continues her clinical practice at Grady Hospital in Atlanta. She is president of the Georgia affiliate of the American College of Nurse Midwives and she directs Emory's nurse midwifery program. Please join me in welcoming Kate. Hello everyone and thanks for coming to my presentation and happy International Day of the Midwife to all of you. Welcome to this discussion about how our program assesses clinical competency in our students. Preparing new nurse midwives and as many as possible is an essential strategy for continued progress towards meeting the World Health Organization Sustainable Development Goal 3, which is to ensure healthy lives and promote well-being for all at all ages and which has a specific focus on reproductive maternal newborn and child health. And while there's great progress being made toward that goal in terms of increasing the percentage of women whose births are attended by a trained attendant and in terms of the health of newborns and children, WHO has indicated that nurses and midwives represent more than 50% of the current shortage of health workers and estimates that the world will need an additional 9 million nurses and midwives by the year 2030. The largest needs-based shortages of nurses and midwives are in Southeast Asia and Africa, but even in the U.S. and particularly in Georgia where I live, half of the counties lack an obstetrician or a certified nurse midwife. So what this means to me as a midwifery program director is that our program has continued to push its capacity while ensuring that students obtain the didactic and clinical opportunities that they need to be competent caregivers. The objectives of this discussion are listed here. We'll talk about what competency-based education is, why this approach is important for the preparation of midwives, then I'll show you a couple of tools we use to ensure that our graduates are competent and that our program is not operating beyond capacity. Competency-based education is a flexible individualized learning schedule focused on a student trajectory of improving performance towards well-defined standardized outcomes. Traditional educational strategies are often very school and teacher-centric and very process-focused. Whereas traditional education often has very structured schedules, sometimes with required readings or required class attendance, there are usually a specified number of clinical hours in which passing criteria may involve a certain number of required procedures and generally involves not scaring anybody. And then the class must pass high-stakes examinations each semester or term and at the end of the program. Many undergraduate and graduate clinical programs are designed this way. But in competency-based education, the focus is on meeting the learning needs of the individual and the timeline and the learning methods are more flexible. The point is for the student to be able to demonstrate progression and then mastery of the acquired skill content and the skills. Competency-based education is student-centered and outcomes focused. We need competency-based education for two main reasons. First, because of the word competency. Midwifery education programs are gatekeepers for the profession and we want to be sure that everybody who graduates from our programs is going to be safe and a wonderful practitioner. And we're not going to let them go and hurt somebody just because they completed some clinical hours and passed a test. While our board exams are effective for assessing knowledge and judgment to a large degree, they don't do a great job of capturing competency involving all of the complexities in a clinical setting, including not only knowledge and judgment, but also manual skills and professional behaviors. The other reason we need competency-based education is that it can help us evaluate students using standardized outcomes despite the wide range of program and student-related factors that influence learning. The range of program factors includes different types of programs. So in the U.S., there's an alphabet soup when it comes to different types of midwives. There are certified nurse midwives, certified midwives, certified professional midwives, community midwives. And then there are different content delivery methods, distance programs versus on-site programs, full-times versus part-time study. And then sometimes there's a huge variance in student clinical opportunities. Some of our sites are busy. Some provide care for higher-risk people. And then sometimes students have good or bad luck as far as whether their call time is busy. We know that the hours don't necessarily match up to the quality of education. And then there are student factors, different learning styles, different work backgrounds, and a variety of life factors affecting how students are able to interact with the didactic material and with clinical practice. Anyway, about it, though, we would really like our students to have the same very basic skill set when they graduate. So as an example, my program at Emory is mostly face-to-face, but there are so many variables that make it impractical to approach a group of students as if each had the same learning needs. Right now, in our first-year cohort, we have 19 students finishing their second to fourth semesters, and they're a very diverse group. As far as prior work experience, 13 just completed their undergraduate nursing degree as a second-degree program. And they come from a variety of prior careers. One was a lactation counselor. Five of the group had prior nursing employment. Four worked in labor and delivery. One was a birth center nurse manager. One worked in the operating room. One worked as a childbirth educator in doula. In addition, there are four completing dual specialties or majors. There are two part-time students. Several have outside employment, and two of our students just had babies. As far as the clinical opportunities available to our students, they're also really variable. Clinical sites may be in the hospital or in a birth center, and these obviously require very different sets of skills and different time investments. Our preceptors are usually midwives, but sometimes there are nurse practitioners or physicians, so lots of variables. So it makes sense that the process of preparation is going to vary in order to meet the needs of each student. However, it's essential that the outcomes for all graduates are consistent, and this leads me to tell you about the tool that we use to help us evaluate those learning outcomes. The Midwifery Clinical Competency Assessment Tool, or MCAT, was developed as part of a faculty quality improvement effort designed to respond to student feedback that they didn't really know how to assess their clinical progress in the program. Students knew they needed to attend clinicals and show that they knew what they were doing, and they had access to some information about core competencies and about how many procedures they should aim to perform, but they wanted more specific goals and criteria that would reassure them that they were on track to be safe competent providers. In addition to trying to do a better job of delineating performance expectations for both students and preceptors and faculty, the MCAT aims to ensure the quality of student midwifery preparation across domains of practice, and for practice directors, the MCAT can help to track the quality and quantity of clinical opportunities across clinical training sites, which is helpful for assessing program capacity. So here are the ingredients used to design the MCAT, and I'll get into a little bit more of each of these on the following slides. I use the International Confederation of Midwives Essential Competencies for Basic Midwifery Practice in 2013, which has since been updated. The World Health Organization Strengthening Midwifery Toolkit has a monitoring midwifery competencies self-assessment tool, and you'll see how our tool looks like that one. ACNM, American College of Nurse Midwives, has core competencies for basic midwifery practice, and they use Patricia Benner's novice to expert model. The basic framework of the MCAT was borrowed from the International Confederation of Midwives Essential Competencies for Basic Midwifery Practice, and shown here are the four categories of the eight updated 2018 competency framework. And here's a sample page from the World Health Organization Strengthening Midwifery Toolkit, Monitoring Midwifery Competencies Self-Assessment Tool. The competencies listed on the left are from ICM. It's a really great list of very specific competencies that are required for safe midwifery practice. This tool, including the format, is from WHO. This is self-assessment tool, and you can see that they use very broad definitions of competence and confidence. I like this tool a lot, but I struggled a little with the broadness of the definition of competence. I wasn't completely comfortable with how knowledge and skills were interchangeable for all of the competencies, and I wanted a more specific indication of the student's level of independence and their ability to deal with complexity. Also, I've worked with a lot of student who's levels of confidence for confidence, for no relationship with their actual skills and abilities, sometimes too much, sometimes too little, so I thought that our tool would be okay without the confidence ratings. So, across reference, the specific competencies of the ICM Essentials with ACNM's list of more general core competencies and made sure that our list, A, covered bowls, and B, was specific to practice in the U.S. Here, you can see some examples of how the ACNM core competencies were matched to ICM Essentials and what those translated into for the MCAT. So, the ACNM component of midwifery care is confirmation and assessment of labor and its progress. The related ICM Essentials are physiology of the stages of labor and indicators of latent and active phases of labor, and the specific competencies we identified for the MCAT are those listed, conducts a thorough interval, subjective history, thoroughly reviews health record, including lab work and diagnostic tests, performs a focused physical exam in labor, including abdominal assessments for fetal position and descent, pelvic exam for dilation of basement descent, presenting part, fetal position, I guess you can read the words. For the self-assessment piece of the tool, I wanted to use a definition of competence that was especially relevant to nursing and midwifery practice. So, I looked to the work of nursing theorist Patricia Benner. In the images on this slide, you can see pictorial summaries of her work in describing the clinical knowledge and behaviors of novices, experts, and everyone in between. In general, you can see that well a novice makes decisions using rules or algorithms to follow in a relatively painstaking way. Experts enjoy a more instantaneous understanding of larger patterns, typically based on the totality of relevant and interacting data rather than on a formal thought process. The MCAT list of competencies reflects some of the complexity of behaviors from Benner's definitions, but they're also based on agreement among the midwifery program's faculty regarding the expected clinical performance of a finishing midwifery student. So, Benner's terminology and our MCAT definitions differ both in substance and in the typical timeframe for achievement. I'm not sure Benner would approve of our definitions, but we're like for instance, for instance where she allows at least two or three years of employment prior to the achievement of competence. Our programs are not necessarily that long. So, the MCAT's definition of competence focuses on the efficiency, flexibility, and independence a student might acquire during their two years or less of midwifery education. I'd hear the definitions we agreed upon that can be best understood if we use an example. So, I'll use the example of a student midwife catching a baby. A novice might need to review hand maneuvers while she washes her hands, and the preceptor might need to talk her through the process or even apply hands over the student's hands. The student took part in the birth, but the precepting midwife was clearly in charge of the situation. An advanced beginner has replayed the birth in her mind 100 times, so if the birth is very straightforward, no help is needed for the hand maneuvers. The student's hand maneuvers are in order and are rather mechanical, and all as well as long as nothing funny happens. The preceptors confirm that it was time for delivery and that the student would try to do the birth without as much assistance and stands by just in case. A competent student knew that even though the mother was 5 centimeters dilated just minutes ago, she better put gloves on when she hears the mother making pushing sounds. She doesn't need prodding to ask the mother to change position if the fetal heartbeat indicates the need, and she can catch the baby in any position. She automatically manages some gushing of blood during the third stage, and her preceptor feels the weight of training lifted. The student is now a helpful person whose presence lighten the preceptor's workload instead of adding more work. A proficient or expert student is beyond that description. She's encountered enough situations that there aren't as many new situations to encounter. She's built up confidence that she can handle problems that arise, and she may serve as an experienced resource to others providing care. So now we can have a look at the MCAT tool. The front page includes the ratings and definitions we've been discussing, and it also tracks didactic coverage of content. So you can see on the top are the definitions, and on the bottom in the table are the courses that we have in our program, both the clinical courses and the courses that are not directly midwifery, but that definitely impact knowledge in school skills related to clinical practice. Each domain is presented separately on the following pages. So the domains are like well woman, antipartum, interpartum, postpartum, newborn, and abortion care. Each domain, so this presentation is just going to show you the first pages for each domain of practice. Some are more than one page. Here's the first page of competencies related to preconception care, family planning, and well woman care. As you look across the page you'll see on the left the competency item, and then the course where content is covered didactically. We wanted to track our content coverage to make sure we were getting it all in. Then there is a column for our first clinical exposure that typically involves some observation in the clinical setting, and then the competency definitions. From top to bottom, you'll see that the top of the form generally has most basic competencies, such as those relating to normal routine assessments, and that the end of the list of items for each domain generally has things that are more like complications. Here you can see abnormal uterine bleeding, polycystic ovarian disease, pelvic pain, etc. We expect that all of the competencies on the top of the page will be rated by the student as at least competent by the end of the program, and that some in the bottom may not because they may not be encountered as regularly in the clinical setting. There are some better descriptions of this in some other domains. So here's the domain having to do with pregnancy care. At the bottom of the page you can see an example of an item in which the student may not become competent managing before graduation and treat uterine fetal death. If a student doesn't have the experience of carrying someone with the fetal demise during the space of their academic program, they should be sure to consult others for assistance after they graduate. From the faculty or program standpoint, we know to include case studies and lab simulations for items like this that are less likely to occur for every student to experience. There are some common items that require competent ratings, such as identification and collaboration, collaborative management, or preeclampsia. Here's the first page of the labor and birth competencies. An example of labor and birth experience that students should not have the opportunity to achieve competence in, within clinical setting at least, is management of cord prolapse. So we definitely hope that they're going to make great progress in most of the columns, most of the rows, I mean, but not all of them. And here's, and the next slide is the first page of the postpartum care items. And there's also one for newborn care. And so that I won't belabor these. Here is the first page of abortion and loss-related care. The last pages of the tool allow for cumulative evaluation of student progress towards competency goals, evaluation of student strengths and future learning goals, and endorsement by students, faculty, and preceptors. As far as logistics, we're filling out the form. Students are instructed to update the entire form prior to each site visit, which is generally once per semester in our program, so that preceptors and faculty can review the form and discuss student progress during the site visit. This is what a completed form might look like after one of the earlier semesters in our program. This student started her interpartum rotation in May, and you can see that she gained a lot of good experience during just that month, because so many of the rows have been filled in with that first clinical exposure and with some level of competency. If I was doing a site visit with this student, I would see that she has a balance of independence and support for some of the more hands-off parts of labor assessments, and that she's more of a novice, not independent at all, when it comes to almost anything that's hands-on. She hasn't had an opportunity for some items. It isn't surprising that she hasn't dealt with a face presentation or a breach birth yet, because as earned as common, but the fact that she didn't fill out anything for nutrition and hydration is something for us to discuss. We would talk about how hospital policies can impact comfort, dignity, and labor outcomes, and how best to navigate those kinds of policies for different laboring women. After using the tool for a few cohorts now, there are some tips. Students complete self-assessments at regular intervals. We use this form each semester during the student site visit. The student fills out the form because it's long, and because part of the point is for them to see where they've made progress and where they need to focus in the future. And then student ratings are verified with preceptor and faculty observers. Some have suggested that they would prefer an electronic version of this form. The paper version has worked well for us. It's easy to review during site visits, and then students are asked to upload a scan copy to our online learning platform. We use Canvas at the end of each semester. However, I am about to start trying to use electronic data entry through SurveyMonkey in order to make the tool a little friendlier to students. It may also help with summarizing the data since the results can be downloaded to an Excel file. Another tip is that the MCAT isn't meant to replace more frequent clinical evaluations or journals. It's really more of a tracking tool that evaluates overall progress, but we use a different evaluation form for more frequent updates, usually at least a couple of times during each semester. I'll show you part of our evaluation tool on the next slide. The third tip is that qualified students may not achieve competence for every item by program end. We've discussed rare opportunities, like core prolapse or abortion-related care, at least rare for midwives in Georgia, that may be better addressed through simulation and didactic instruction. So just FYI, this is a part of the tool that we use at intervals during the semester to facilitate communication between the student, the preceptor, and the faculty. You can see the rating scale on the bottom is the same as the one we used for the MCAT, but that the clinical objectives in the middle of the page are much more general. There are some limitations to the tool. They may also be understood as challenges. The length of the tool, it says 10 pages, it's really more than that, can seem overwhelming at first to students and preceptors, but once they start filling it out, it's really not as bad as it looks since the items are so familiar. We'll still see if SurveyMonkey helps with this process. Second, the self-assessments are subject to over and under estimates. We discussed how some students may over-underestimate their performance. Some of the ratings may need to be edited during the site visit to make them more realistic. And then in my last cohort, I had the experience of a student trying to manipulate the system using her ratings. She wanted to have extra clinical experience, so she rated herself lower than was necessary on many items. We needed to have a conversation about trust, and I needed to assure her that she wouldn't be set free from her program until she had the opportunities to increase her competencies in key areas. The third main challenge is determining realistic and adequate expectations for domain-level for your program. So for example, a goal for each during semesters one and two might be that they achieve 70 percent of the ambulatory competencies will be scored as advanced beginner or above. And then in semester three, 70 percent of well-woman and prenatal competencies will be scored competent or above, and 70 percent of interpartum or postpartum competencies will be scored advanced beginner or above. By our last semester, semester four, we would like at least 90 percent of the competencies to be scored as competent above or above, although I recognize that other programs may have other preferences for student goal setting. So that's the MCAT, and now we'll just talk briefly about the semester summary tool. So the semester summary tool is like partner tool for the MCAT, and it's used to tally the data from the whole student cohort. So this is the first page, and it summarizes performance over the first two semesters of our program. Our program has lab simulations in the first semester, but no actual clinical work. So those are combined for the sheet. You can see that each page can be used to see each student's course grade, their clinical competency levels by domain and overall, and then other bits of data like birth numbers and clinical hours. The numbers under each domain column are the number of MCAT items in which the student achieved an adequate degree of competency. So for the semester pictured, faculty wanted students to achieve at least 70 percent of the MCAT items as novice or above. The color coding shows how well each student did overall. So the green means that students achieved at least 70 percent novice overall as we were hoping. Yellow is for students achieving between 50 and 69 percent, and red is for under 50 percent. The student participated in a few clinical hours and was exposed to fewer experiences. It can be used by program directors to get a view of an individual's performance and of the cohort's performance, and it can identify gaps in the program or in the program's clinical opportunities. For example, our program found that students were not getting much exposure to loss and abortion, so we knew that we needed to add in some case studies and some simulation around those items on the MCAT. Also, some clinical sites offer more mostly pregnancy related care and little exposure of students to gynecological concerns. It can also be used to determine program capacity. Students really should be green, and if they aren't due to program versus student factors, then the program director needs to take another look at how many clinical spaces they have to offer their next cohort. And that is what I wanted to tell you, so I'm happy to take questions. Okay, I have some questions for you. Okay. First, a comment that Catherine made toward the beginning of your presentation. She said that when you were talking about board exams and how they don't assess all skills, she said they also don't assess effective skills very well. Do you want to make any comment on that? I completely agree. I will say that in the second, in the new update from the ICM Essentials, they actually took out the effective skills because they're so subjective and very hard to evaluate. So, I'm not sure that I've figured out anything better for that either. All right. Sheila asked, is self-assessment done by self-peer, self-peer, clinician, or educator? It is done by the self, the students evaluate themselves, but then their evaluations should be reviewed with both their preceptor, and we encourage them to make sure that they review it with preceptors who they've spent the most time with and also with their faculty site visitors. Right. And Lorraine made the comment agreeing with you, self-assessments are notoriously over or underestimated. Yeah. They sure are. Definitely. I've had the best students in the class saying that they just, they weren't sure. I think that they were rating their confidence or they just, they were bringing their confidence into the evaluation. And what I do in those cases is go back with them to the definitions. And if they really read through, can you do this as long as everything is straightforward and your preceptor is standing there, then their advanced beginner, they're not novice anymore. And just some students are really modest. And then there are some who rate themselves as expert on way more things. And Lorraine just commented exactly it is often about confidence. So it really is. Donna had a comment that the paperwork that you were showing is similar to the competencies that they use in the UK. So I thought that was interesting. Really? That's cool. I was actually really interested in hearing more about how people would use something like this and they're studying what would be, you know, some of the differences, because I know that our midwifery might vary so much between countries, with our education being nurse midwifery, you know, it's things that I would love to learn more about. Okay. And then Catherine asked if any of Emery's programs use Typhon? We do use Typhon. And I had asked if we could put this into Typhon. And the answer was no. And I'm trying to remember why they told me no. I think there were, I don't remember why. I think it was too many questions or too many competencies. Not sure. Okay. And Catherine also had a question about primary care. And Catherine, I'm not sure what your question was. I believe you're asking if primary care skills are assessed for competency. Is that what you're trying to ask? That's what I was thinking when I saw her question. And I think that they're integrated somewhat into the preconception, family planning and well women care. It's definitely very broadly though. Okay. So she says on the scoring sheet, I don't see a competency for primary care. It's not a separate section. That's true. Okay. So you have it integrated. And Macon makes the comment. We use very similar competencies at MCU, which I believe is that Midwives College of Utah for CPM Midwifery. Very cool. And she says yes. And Catherine made another comment that there's a lack of opportunity in many programs. I believe she's still commenting on primary care. I agree. And sometimes students don't recognize their opportunities for primary care. With a lot of the domains or competencies, we really need to like people don't necessarily know that they're providing preconception care either until you really explain what that is. And if you ask somebody whether they plan to become pregnant in the next year and then try to help them work out the answer to that question in a practical way either to promote say pregnancy or prevent pregnancy, that's preconception care. So they don't know to check that box until they have a better understanding of what it is. And primary care tends to not be a separate visit for the students in midwifery settings. So they have to understand that when they're trying to determine if a common discomfort of pregnancy is asthma, that counts as a primary care visit. And Margaret asks about home birth. I believe maybe asking if there's any assessment of home birth competency. I will say I wish I could say that we had that and we don't. It is, I was just attending a session for these meetings that was on home birth. That was so interesting. And we are way short on home birth. In Georgia where I live, there is very little home birth and well, certified nurse midwives, which are the only kinds of midwives that are licensed in our state. Certified nurse midwives may perform home births, but CPMs are not even licensed in Georgia. So home birth is not common enough for it to be something that we offer routinely in our program. But it would definitely hopefully be included for other programs. It's a lean comment. It's sad. Definitely. Yeah, we have a lot of work to do for our scope of practice in Georgia. And for the school is part of that driven by insurance because I know with teaching for frontier, our student malpractice insurance will not cover for them to attend a home birth even as an observer. That is also an issue for sure. I think there are a lot of challenges for us when I think of being able to offer home birth as much as I think it would be important and really add a lot to the future practice of students to be able to attend home births. It would be there aren't enough of them here to make it a practical requirement. It could be something that when someone wants to attend, they would need to do it outside of student insurance as you're saying. But there, most of the students are not getting so much, so many clinical experiences that they have a lot of extra time to be on call for an individual person's home birth. I know that's so sad. I hate saying that. Yeah. But I think that we would have to have a little more space in our program for, I don't know, I hate to even use the word extra there because I think it's essential. It's just not realistic for us to offer it. And Cami from New Zealand said that's hard with no insurance for students attending home birth. And Tara, Tara Matty, the students in my country would have clinical attachment at primary care facilities. Oh, that's nice. That's great. Right now we have a dual, a lot of students doing a dual specialty. So they have a lot of primary care time. We're in the process of phasing that out. And then we're really going to have to come up with some more opportunities that are focused on primary care for sure. All right. Are there any other questions that I have missed or that have not come in yet? Okay. I believe that's everyone. Oh, no, here came another one. How does your MCAT help with accreditation? Well, actually, we put together the MCAT the year that we were re-accrediting. And it was super helpful. It's been helpful a number of times when we were trying to, I mean, so many questions, it's handy for answering, whether it's how do we measure student competence or what opportunities do our students have? I can just show them the numbers. So it was very helpful with accreditation. Well, that sounds like an excellent asset to have. I really liked your forms. Thank you. All right. It looks like we are at the end of our questions. So I am going to stop the recording. And thank you so much, Kate, for this excellent presentation. It's been very valuable. Well, thank you all for attending and for helping to make this work and for your great questions and comments.