Transcript
Melanie Barlow: All right, welcome to another episode and I'm very excited today. i' am joined by two academic colleagues. We have Kate Dawson, Associate Professor of Midwifery, who is also National Course Coordinator for Midwifery at the Australian Catholic University, and Lanelle Moran, who is the National Specialised Learning Environments and Simulation Coordinator for Midwifery and also Lecturer in Midwifery. So welcome.
Lynnelle Moran: Thanks for having us Mel.
Kate: Thanks for having us now. It's great to be here.
Melanie Barlow: Thanks. Now, I came across your work because you presented a sim showcase in December last year.
Kate: Yeah.
Melanie Barlow: And you were talking about your topic was the learning matters, exploring student experiences of a formative slash summative OSCE assessment.
Melanie Barlow: So I thought that was interesting. So can you tell us a bit about that and why you wanted to go there?
Kate: yeah i I guess the project was first initiated during COVID. I don't think that actually made much difference, the timing of it. But I guess we we really noticed and I think probably everyone does that when students present to their OSCE assessment, so objective, structured, you know, an assessment of skills,
Kate: some students come in really visibly nervous, shaking, can't get their words out. it Like it it really created a barrier to, i think, how they perform in the OSCE and are they able to demonstrate their knowledge and skills when they're in that really heightened state of anxiety.
Kate: so I was like, okay, how can we, address this? Is there a way to do this that reduces anxiety and then potentially creates more of a learning experience for the students?
Kate: So I think that was the initiative to start with. Okay, how can we reduce this? So I thought thought maybe if we introduce formative, gave them an opportunity to practice that or kind of practice, got feedback and then came back, would that make a difference to that anxiety level and then learning?
Lynnelle Moran: And we all recognise that OSCE is a great assessment tool for midwifery because it's relational and clinical in terms of what it's asking the students to demonstrate. And it's a team dynamic often, or in this context, it was a team exercise. So it's a really beneficial assessment tool. And in this current climate where we're all scrambling to try and find, you know, assessment tools that really based on student knowledge and learning and capacity. It's just a fantastic tool, but we recognised this heightened anxiety that was really inhibiting for some students.
Lynnelle Moran: So, yeah, it was really great opportunity.
Melanie Barlow: So there's a few things there. So when you say, so it was initially a summative assessment. So what were you assessing? Because you were saying team, not individual.
Kate: Yeah, we have a number of OSCEs throughout the course. I think virtually every semester throughout the course we have OSCEs. Initially students do an individual one. So this subject was exploring complexities during childbearing. particularly labour and birth. So students within this class worked as a group throughout the semester and they worked because there's in emergencies or in complex scenarios, it involves more than one midwife, a multidisciplinary team sometimes, but also more than one midwifery staff member in navigating the space, whether it's a an obstetric emergency or hospital
Kate: pre-eclampsia, working out, like if someone's got diabetes and we need to titrate their sugars during labour, there's lots of different people involved. So throughout the semester, these students work as a group.
Kate: They move around the roles during the semester. So each week someone gets to be in charge or the primary midwife and then delegates and communicates with everyone else.
Kate: So At the end, the OSCE is also framed around that. They work with the same group throughout the semester and then in the OSCE. So also we're looking at teamwork, communication, delegation, but they also need to know what they're doing. And we know probably similarly in other disciplines, but in midwifery as well, that emergency scenarios are practiced together a lot but to become familiar with what your role is in those things, how to delegate roles and also communication within that. So there was a lot of different aspects within this OSCE to look at.
Lynnelle Moran: Absolutely. And the two main focus scenarios were around a postpartum hemorrhage response, identification, response and management, and also or shoulder dystocia, recognition, response and management.
Melanie Barlow: So I really like that because often in OSCEs, the clinical skill is spelled out in detail. And then the communication element is just communicated with patient tick, like it's not spelled out in the same detail. So for you, was communication and that delegation and teamwork just as much part of the OSCE as the clinical skills?
Lynnelle Moran: Yeah, it's a fundamental part of it, wouldn't you say, Kate?
Kate: Yeah, absolutely. And it really, I think i think this formative summative OSCE scenario could be applied absolutely to other OSCE scenarios. but particularly for this one and students will feed back after they've done a group, asking, oh, we got that wrong and no one else said anything, like no one else said, hey, how about you do this?
Kate: So it's also, you know, that team dynamic within that that space is really important and the communication and they will often say later, oh, or the debrief sometimes. I don't know, Flanelle, you can speak to that, particularly a debrief we say,
Kate: know, you can talk to each other about, prompt each other about what you're thinking. Yeah.
Lynnelle Moran: Yeah. Yeah. yeah
Melanie Barlow: So what you're talking about, just for clarity, is they used to come straight into the OSCE. high anxiety levels because of it's a summative assessment. So what you did was brought the students in, they did a scenario together, but it was just formative learning.
Kate: Yeah.
Melanie Barlow: So feedback, reflective conversations.
Kate: Yeah.
Melanie Barlow: And then they went out and then they came back same day or different day to do the OSCE.
Lynnelle Moran: Different day.
Melanie Barlow: Different day, but same team.
Lynnelle Moran: Mm-hmm. And same scenario.
Melanie Barlow: And different scenario, same scenario.
Lynnelle Moran: Yeah.
Melanie Barlow: Wow. And so, okay, there's a lot to unpack there. So how, what were the students most anxious about? Like when you say high anxiety, was it the summative element or was it the topic or was it the skill or the teamwork or like, what do you think they were most anxious about?
Kate: you want to comment on them?
Lynnelle Moran: Yeah, well, i think I think when it's a summative and it's a hurdle task, their stakes are so high in the performance of that, in the group performance, in their, they would feel like they'd done all the work to feel prepare prepared. They would have really invested heavily, I would say. Our student cohort is very committed.
Lynnelle Moran: And so I would feel like generally they'd come in very well prepared. you know they would have had compulsory attendance throughout the semester so they would have done all the actual practical skill development under close supervision and in partnership with the teaching team so they've done the work but then they come into this very high stakes environment that they're basically going to end with a pass or fail at the end of that and the consequences of that are great they may not be able to go well they It might impact their attendance on placement.
Lynnelle Moran: It will certainly impact their ability to complete you know the unit successfully. And so it means that they'll need to repeat the unit. So there was so much riding on that performance and interpretation and responsiveness to a scenario that I think it's with good reason that they were presenting with such high anxiety.
Melanie Barlow: Mm-hmm.
Lynnelle Moran: Anything else to add to that, Kate?
Kate: No, but just triggered a thought in, you know, these students within our standards, midwifery standards, we say we need to do some sort of assessment before students go on placement to ensure that they're safe to go out into the real world and practice on on people.
Kate: So if we're doing a high stakes assessment and it's pass or fail and they come in nervous and it's not a learning experience, then what that how does that add value to them going on placement?
Kate: If we can create a scenario where they're coming in practising those skills, developing the muscle memory around how to communicate, how the dexterity around what to do, and we're we're enabling them to give to give them feedback and then repractice that with the feedback, surely that's going to create a safer student to go then out on into the,
Melanie Barlow: Yes. yes
Kate: clinical world and practice on real people or or refine those skills on real people and and be more cognizant of the i emergencies as they emerge and how to react to that.
Kate: So there's that element as well, I think, in terms of the motivations.
Lynnelle Moran: And I think, Kate, you know, this was such a creative and innovative way to look at the student learning experience and to go to to be cognizant of the psychological safety of the students in that moment.
Lynnelle Moran: and to say, wanna create learning environments where these students can thrive and flourish and to set them up for success when they go out on placement. And this, we we just were witnessing something that was really in conflict with that.
Lynnelle Moran: And so I love that Kate saw this as an opportunity, even though it meant a lot of investment of resources and staffing and a re-imagining of how we normally did things.
Lynnelle Moran: I think it was really the heart of it and the driving force behind it was the wellbeing of the students and to enrich the learning experience, which is, you know, the end end goal, isn't it?
Lynnelle Moran: So it was a great idea.
Melanie Barlow: That's a good, so it's not just about ticking a box for assessment, it's genuinely making sure the students are practice ready, safe learning and good for their wellbeing.
Lynnelle Moran: though
Melanie Barlow: Excellent. So what did you see then afterwards? What about performance and their anxiety levels after you implemented the formative?
Kate: Yeah, definitely. i mean, we, wait in this study, we assessed anxiety levels, pre-formative, post-formative, pre-summative and then after everything. And in our surveys we had lots of open-ended questions to elaborate on that.
Kate: And anecdotally, a lot of students afterwards said, oh, yeah, we knew that was happening and it was so good. I'm like, well, why didn't you participate in the study?
Kate: And they're like, oh, we just was we're under so much time pressure. We didn't feel like we could at the time, but can you do it again so we can now input? I'm like, no, no, that's two that's past. or if We could do another version in the future.
Kate: of I've lost track of the question in the first.
Melanie Barlow: Yeah, so anxiety levels for those that did the formative.
Kate: Oh, yeah.
Lynnelle Moran: Yeah. Kate, I've got a beautiful quote that I could share from one of the students who participated.
Kate: Yeah, great.
Lynnelle Moran: and this is from the in-between moment. You know, this is the the interval between the formative and the summative. And the student says, I feel an overwhelming sense of relief just knowing what worked well and what we need to work on as a group.
Lynnelle Moran: I'm feeling so much better about the upcoming OSCE assessment, feeling confident and significantly less stressed. The tutors made us feel so comfortable being relaxed in their demeanour smiling and being personable. They gave us opportunity to express what we felt we'd done well and what we felt we didn't do as well. They provided great feedback as so we know what we could improve.
Melanie Barlow: Lovely.
Kate: Nice.
Lynnelle Moran: Yeah.
Melanie Barlow: yeah So how did you measure anxiety?
Kate: Yeah, we used the State Trait Anxiety Questionnaire. There's a few different versions. There's like a 40 question version, but I felt like that was a lot. and So there was a 20 question version, which is well used in the literature around measuring anxiety.
Kate: So there's a series of 20 questions that the students tick, like they have a Leichhardt type scale and they tick where they they're feeling on that anxiety sort of level.
Kate: So yeah, we measured it using that again in the pre and post formative and then post summative, so three time points throughout the the study.
Kate: And they were obviously administered fairly close together. We we did the formative and summative OSCE about a week apart, so it gave them opportunities to practice and regroup as a group for anything they wanted to improve. So we administered that carefully in between.
Kate: And we saw a real a definite decrease in the overall anxiety level between pre-everything and then the post-formative And then also another decrease in the anxiety level between, until after the summative, but not as significant, like there wasn't a big drop there. The the biggest drop was between the first, everything and know, doing that practice as the quote that Lanell read out sort of reflects, that, that reduced the anxiety significantly.
Kate: and then went on to a slight further reduction but not as great as that first one.
Melanie Barlow: Excellent. Have you written this up?
Kate: Yeah, we've just submitted a paper for publication, so fingers crossed.
Melanie Barlow: Excellent.
Melanie Barlow: Great. So let us know when that's published and we can put a link in for our listeners.
Lynnelle Moran: Thanks, man.
Melanie Barlow: So have you implemented this more broadly now based on the results?
Kate: It's we implemented it into our third year, first semester subject where they do an OSCE in a smaller group, but related to neonatal resuscitation.
Kate: And I think we could implement it earlier as well. I think it it would be beneficial to implement it earlier. So at the moment, just in that third year, first semester one.
Kate: Yeah.
Lynnelle Moran: But i think it's I think the learnings of it have informed how we approach OSCEs throughout the program. I know because I work on several of the professional practice units and it's made me much more mindful of the the heightened anxiety and, you know, from the feedback that we receive from students through this these open-ended questions especially. just how, what a barrier that was to their performance and preparation even. So yeah, I think we've talked a lot about that as a faculty and just ways to mitigate that across the board.
Melanie Barlow: what do you think then, based on these results and studies and the quotes you've received, like, what does that say about our approach to you know, these practical assessments? Because I'm thinking... in the advent of AI and we need more secure assessments, simulation has to come to the fore because it's a secure form of assessment. So if people are looking to OSCEs more for that secure assessment, what would you recommend and like what does that mean for our students going forward with these high-risk assessments?
Kate: I guess there's always lots of considerations in that and I guess in the background we're thinking about staffing and funding and can we do that but are just reflecting back to an earlier comment of you know we need to make sure these students are practice ready for going on placement we know it's a secure assessment you could incorporate You know, if it was an individual assessment, you could incorporate more of an oral aspects and make sure students are talking through what they're doing and have criteria around that.
Kate: But I think because it's key knowledge that they have to know, I think feedback is essential. It can't be, know, I have a bit of an issue with the exams in that we're testing key knowledge, but then students don't get the feedback about the areas that they're and missing or that they need to strengthen
Kate: in that And it goes, it's the same sort of concept with a purely summative OSCE, if students are coming in and not getting the feedback and not understanding. Obviously, if things are really wrong, they'll ask be asked to come and do a re-sit and perhaps, and we know when students in midwifery anyway come back to do a re-sit at any point that they often nail it the second time. So that's also indicative of that feedback and the value of the feedback that students receive at that point to then come back and know what they're doing in that second instance.
Melanie Barlow: That's so true.
Lynnelle Moran: And I think these positive experiences, when that feedback, when positive feedback is modelled by the preceptors or by the teaching team, those students take that into their practice as well. And I think it's so generative and it's really important because so often it's punitive. You know, the hurdle task is pass or fail or, you know, feedback in the clinical setting, you know, once they graduate, it's, it's, very reactionary and punitive. No one has time to be preventative or look at, you know, areas that require development. And so I think it's a symptom of the current landscape and we can help students to develop those skills by modeling that through these assessment processes as well.
Lynnelle Moran: And they will take it into practice.
Melanie Barlow: Yeah, very good. So I guess to wrap up, if people are listening to this and they want to implement it in their own areas, what's a bit of advice?
Kate: I think it's pretty easy to implement. So I think give it a go and see what, see, listen to the student feedback and listen to what they're saying, because I'm sure the students will really appreciate it.
Kate: And it's basically a repeat of the OSCE you've already got set up, but giving students the opportunity for a formative aspect of that and then a summative.
Kate: Obviously staffing's important and and those resources, So to build that capacity in to where you're going.
Melanie Barlow: Did you need to do any extra faculty development? Like what is the difference between a formative group, immersive simulation versus summative and OSCE and the role they play?
Kate: We didn't really, you definitely could do that. I think the team was pretty on board with, like we set that up well, like this is how it's structured and and this is the process, so set up the process really clearly.
Kate: Could definitely talk to staffing around formative assessment and I think the debrief in the middle and how to give that constructive feedback in the middle is important. I'm sure, Mel, you can talk to that. You're probably expert in that area more than I am, absolutely.
Kate: But that That's also a really important part to be able to give that really constructive feedback in the middle.
Melanie Barlow: Yeah, very good. Excellent. so we ask all our guests this because we are Simulation Happy Hour. I have my coffee in front of me, which is regular. But if you could be and Happy Hour anywhere in the world, where would you be?
Melanie Barlow: And which what would you be drinking?
Lynnelle Moran: Oh.
Melanie Barlow: It could be anywhere from like, you know, in Italy with an espresso through to the beach with a
Lynnelle Moran: I love this question, Mal. I want to be in Iceland with a spiced rum.
Melanie Barlow: Nice. How
Kate: I've been really thinking about the Netherlands and Amsterdam lately. I love i love that place. So there it's probably cold there now. So Avam Chocolat, like a hot chocolate with cream on top.
Kate: It's very good.
Melanie Barlow: All right, take me to both those places. That's amazing.
Kate: Love the question.
Melanie Barlow: So Lynnelle and Kate, thank you so much for joining us today. And, yeah, I think that's a really practical activity that people can implement and, you know, create change straight away, it sounds like.
Kate: Thanks Mel, thanks for having us.
Lynnelle Moran: lovely, Esmel.
Melanie Barlow: Pleasure. Pleasure.
