Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
It's the learning that matters
25 Plays2 months ago
This month we welcome the discussion with two midwifery academics, Associate Professor Kate Dawson and Lynnelle Moran on the topic: It's the learning that matters - exploring student experiences of a formative- summative OSCE assessment. Their study focused on exploring feelings of anxiety and psychological safety as well as learning experience of students throughout the assessment period, including a summative OSCE assessment.
Recommended
Transcript
Intro
Introduction of Guests and Topic
00:00:40
Melanie Barlow
All right, welcome to another episode and I'm very excited today. i' am joined by two academic colleagues. We have Kate Dawson, Associate Professor of Midwifery, who is also National Course Coordinator for Midwifery at the Australian Catholic University, and Lanelle Moran, who is the National Specialised Learning Environments and Simulation Coordinator for Midwifery and also Lecturer in Midwifery. So welcome.
00:01:09
Lynnelle Moran
Thanks for having us Mel.
00:01:11
Kate
Thanks for having us now. It's great to be here.
00:01:14
Melanie Barlow
Thanks. Now, I came across your work because you presented a sim showcase in December last year.
00:01:21
Kate
Yeah.
00:01:22
Melanie Barlow
And you were talking about your topic was the learning matters, exploring student experiences of a formative slash summative OSCE assessment.
00:01:32
Melanie Barlow
So I thought that was interesting. So can you tell us a bit about that and why you wanted to go there?
Origins of the OSCE Project
00:01:40
Kate
yeah i I guess the project was first initiated during COVID. I don't think that actually made much difference, the timing of it. But I guess we we really noticed and I think probably everyone does that when students present to their OSCE assessment, so objective, structured, you know, an assessment of skills,
00:02:06
Kate
some students come in really visibly nervous, shaking, can't get their words out. it Like it it really created a barrier to, i think, how they perform in the OSCE and are they able to demonstrate their knowledge and skills when they're in that really heightened state of anxiety.
00:02:30
Kate
so I was like, okay, how can we, address this? Is there a way to do this that reduces anxiety and then potentially creates more of a learning experience for the students?
00:02:45
Kate
So I think that was the initiative to start with. Okay, how can we reduce this? So I thought thought maybe if we introduce formative, gave them an opportunity to practice that or kind of practice, got feedback and then came back, would that make a difference to that anxiety level and then learning?
Importance of OSCE in Midwifery
00:03:09
Lynnelle Moran
And we all recognise that OSCE is a great assessment tool for midwifery because it's relational and clinical in terms of what it's asking the students to demonstrate. And it's a team dynamic often, or in this context, it was a team exercise. So it's a really beneficial assessment tool. And in this current climate where we're all scrambling to try and find, you know, assessment tools that really based on student knowledge and learning and capacity. It's just a fantastic tool, but we recognised this heightened anxiety that was really inhibiting for some students.
00:03:48
Lynnelle Moran
So, yeah, it was really great opportunity.
00:03:52
Melanie Barlow
So there's a few things there. So when you say, so it was initially a summative assessment. So what were you assessing? Because you were saying team, not individual.
00:04:03
Kate
Yeah, we have a number of OSCEs throughout the course. I think virtually every semester throughout the course we have OSCEs. Initially students do an individual one. So this subject was exploring complexities during childbearing. particularly labour and birth. So students within this class worked as a group throughout the semester and they worked because there's in emergencies or in complex scenarios, it involves more than one midwife, a multidisciplinary team sometimes, but also more than one midwifery staff member in navigating the space, whether it's a an obstetric emergency or hospital
00:04:49
Kate
pre-eclampsia, working out, like if someone's got diabetes and we need to titrate their sugars during labour, there's lots of different people involved. So throughout the semester, these students work as a group.
00:05:03
Kate
They move around the roles during the semester. So each week someone gets to be in charge or the primary midwife and then delegates and communicates with everyone else.
00:05:13
Kate
So At the end, the OSCE is also framed around that. They work with the same group throughout the semester and then in the OSCE. So also we're looking at teamwork, communication, delegation, but they also need to know what they're doing. And we know probably similarly in other disciplines, but in midwifery as well, that emergency scenarios are practiced together a lot but to become familiar with what your role is in those things, how to delegate roles and also communication within that. So there was a lot of different aspects within this OSCE to look at.
00:05:54
Lynnelle Moran
Absolutely. And the two main focus scenarios were around a postpartum hemorrhage response, identification, response and management, and also or shoulder dystocia, recognition, response and management.
00:06:11
Melanie Barlow
So I really like that because often in OSCEs, the clinical skill is spelled out in detail. And then the communication element is just communicated with patient tick, like it's not spelled out in the same detail. So for you, was communication and that delegation and teamwork just as much part of the OSCE as the clinical skills?
00:06:37
Lynnelle Moran
Yeah, it's a fundamental part of it, wouldn't you say, Kate?
00:06:40
Kate
Yeah, absolutely. And it really, I think i think this formative summative OSCE scenario could be applied absolutely to other OSCE scenarios.
Challenges of High-Stakes Assessments
00:06:53
Kate
but particularly for this one and students will feed back after they've done a group, asking, oh, we got that wrong and no one else said anything, like no one else said, hey, how about you do this?
00:07:06
Kate
So it's also, you know, that team dynamic within that that space is really important and the communication and they will often say later, oh, or the debrief sometimes. I don't know, Flanelle, you can speak to that, particularly a debrief we say,
00:07:22
Kate
know, you can talk to each other about, prompt each other about what you're thinking. Yeah.
00:07:29
Lynnelle Moran
Yeah. Yeah. yeah
00:07:30
Melanie Barlow
So what you're talking about, just for clarity, is they used to come straight into the OSCE. high anxiety levels because of it's a summative assessment. So what you did was brought the students in, they did a scenario together, but it was just formative learning.
00:07:40
Kate
Yeah.
00:07:47
Melanie Barlow
So feedback, reflective conversations.
00:07:51
Kate
Yeah.
00:07:51
Melanie Barlow
And then they went out and then they came back same day or different day to do the OSCE.
00:07:57
Lynnelle Moran
Different day.
00:07:58
Melanie Barlow
Different day, but same team.
00:08:00
Lynnelle Moran
Mm-hmm. And same scenario.
00:08:01
Melanie Barlow
And different scenario, same scenario.
00:08:04
Lynnelle Moran
Yeah.
00:08:05
Melanie Barlow
Wow. And so, okay, there's a lot to unpack there. So how, what were the students most anxious about? Like when you say high anxiety, was it the summative element or was it the topic or was it the skill or the teamwork or like, what do you think they were most anxious about?
00:08:32
Kate
you want to comment on them?
00:08:33
Lynnelle Moran
Yeah, well, i think I think when it's a summative and it's a hurdle task, their stakes are so high in the performance of that, in the group performance, in their, they would feel like they'd done all the work to feel prepare prepared. They would have really invested heavily, I would say. Our student cohort is very committed.
00:08:56
Lynnelle Moran
And so I would feel like generally they'd come in very well prepared. you know they would have had compulsory attendance throughout the semester so they would have done all the actual practical skill development under close supervision and in partnership with the teaching team so they've done the work but then they come into this very high stakes environment that they're basically going to end with a pass or fail at the end of that and the consequences of that are great they may not be able to go well they It might impact their attendance on placement.
00:09:28
Lynnelle Moran
It will certainly impact their ability to complete you know the unit successfully. And so it means that they'll need to repeat the unit. So there was so much riding on that performance and interpretation and responsiveness to a scenario that I think it's with good reason that they were presenting with such high anxiety.
00:09:35
Melanie Barlow
Mm-hmm.
00:09:48
Lynnelle Moran
Anything else to add to that, Kate?
00:09:51
Kate
No, but just triggered a thought in, you know, these students within our standards, midwifery standards, we say we need to do some sort of assessment before students go on placement to ensure that they're safe to go out into the real world and practice on on people.
Creating Supportive Learning Environments
00:10:11
Kate
So if we're doing a high stakes assessment and it's pass or fail and they come in nervous and it's not a learning experience, then what that how does that add value to them going on placement?
00:10:24
Kate
If we can create a scenario where they're coming in practising those skills, developing the muscle memory around how to communicate, how the dexterity around what to do, and we're we're enabling them to give to give them feedback and then repractice that with the feedback, surely that's going to create a safer student to go then out on into the,
00:10:24
Melanie Barlow
Yes. yes
00:10:50
Kate
clinical world and practice on real people or or refine those skills on real people and and be more cognizant of the i emergencies as they emerge and how to react to that.
00:11:03
Kate
So there's that element as well, I think, in terms of the motivations.
00:11:08
Lynnelle Moran
And I think, Kate, you know, this was such a creative and innovative way to look at the student learning experience and to go to to be cognizant of the psychological safety of the students in that moment.
00:11:21
Lynnelle Moran
and to say, wanna create learning environments where these students can thrive and flourish and to set them up for success when they go out on placement. And this, we we just were witnessing something that was really in conflict with that.
00:11:36
Lynnelle Moran
And so I love that Kate saw this as an opportunity, even though it meant a lot of investment of resources and staffing and a re-imagining of how we normally did things.
00:11:47
Lynnelle Moran
I think it was really the heart of it and the driving force behind it was the wellbeing of the students and to enrich the learning experience, which is, you know, the end end goal, isn't it?
00:11:58
Lynnelle Moran
So it was a great idea.
00:11:59
Melanie Barlow
That's a good, so it's not just about ticking a box for assessment, it's genuinely making sure the students are practice ready, safe learning and good for their wellbeing.
00:11:59
Lynnelle Moran
though
00:12:17
Melanie Barlow
Excellent. So what did you see then afterwards? What about performance and their anxiety levels after you implemented the formative?
Benefits of Formative Assessments
00:12:28
Kate
Yeah, definitely. i mean, we, wait in this study, we assessed anxiety levels, pre-formative, post-formative, pre-summative and then after everything. And in our surveys we had lots of open-ended questions to elaborate on that.
00:12:49
Kate
And anecdotally, a lot of students afterwards said, oh, yeah, we knew that was happening and it was so good. I'm like, well, why didn't you participate in the study?
00:13:00
Kate
And they're like, oh, we just was we're under so much time pressure. We didn't feel like we could at the time, but can you do it again so we can now input? I'm like, no, no, that's two that's past. or if We could do another version in the future.
00:13:17
Kate
of I've lost track of the question in the first.
00:13:21
Melanie Barlow
Yeah, so anxiety levels for those that did the formative.
00:13:22
Kate
Oh, yeah.
00:13:25
Lynnelle Moran
Yeah. Kate, I've got a beautiful quote that I could share from one of the students who participated.
00:13:30
Kate
Yeah, great.
00:13:30
Lynnelle Moran
and this is from the in-between moment. You know, this is the the interval between the formative and the summative. And the student says, I feel an overwhelming sense of relief just knowing what worked well and what we need to work on as a group.
00:13:45
Lynnelle Moran
I'm feeling so much better about the upcoming OSCE assessment, feeling confident and significantly less stressed. The tutors made us feel so comfortable being relaxed in their demeanour smiling and being personable. They gave us opportunity to express what we felt we'd done well and what we felt we didn't do as well. They provided great feedback as so we know what we could improve.
00:14:06
Melanie Barlow
Lovely.
00:14:07
Kate
Nice.
00:14:07
Lynnelle Moran
Yeah.
00:14:08
Melanie Barlow
yeah So how did you measure anxiety?
00:14:15
Kate
Yeah, we used the State Trait Anxiety Questionnaire. There's a few different versions. There's like a 40 question version, but I felt like that was a lot. and So there was a 20 question version, which is well used in the literature around measuring anxiety.
00:14:34
Kate
So there's a series of 20 questions that the students tick, like they have a Leichhardt type scale and they tick where they they're feeling on that anxiety sort of level.
00:14:47
Kate
So yeah, we measured it using that again in the pre and post formative and then post summative, so three time points throughout the the study.
00:14:59
Kate
And they were obviously administered fairly close together. We we did the formative and summative OSCE about a week apart, so it gave them opportunities to practice and regroup as a group for anything they wanted to improve. So we administered that carefully in between.
00:15:18
Kate
And we saw a real a definite decrease in the overall anxiety level between pre-everything and then the post-formative And then also another decrease in the anxiety level between, until after the summative, but not as significant, like there wasn't a big drop there. The the biggest drop was between the first, everything and know, doing that practice as the quote that Lanell read out sort of reflects, that, that reduced the anxiety significantly.
00:15:56
Kate
and then went on to a slight further reduction but not as great as that first one.
00:16:04
Melanie Barlow
Excellent. Have you written this up?
00:16:06
Kate
Yeah, we've just submitted a paper for publication, so fingers crossed.
00:16:10
Melanie Barlow
Excellent.
00:16:14
Melanie Barlow
Great. So let us know when that's published and we can put a link in for our listeners.
00:16:20
Lynnelle Moran
Thanks, man.
00:16:23
Melanie Barlow
So have you implemented this more broadly now based on the results?
00:16:29
Kate
It's we implemented it into our third year, first semester subject where they do an OSCE in a smaller group, but related to neonatal resuscitation.
00:16:43
Kate
And I think we could implement it earlier as well. I think it it would be beneficial to implement it earlier. So at the moment, just in that third year, first semester one.
00:16:58
Kate
Yeah.
00:16:59
Lynnelle Moran
But i think it's I think the learnings of it have informed how we approach OSCEs throughout the program. I know because I work on several of the professional practice units and it's made me much more mindful of the the heightened anxiety and, you know, from the feedback that we receive from students through this these open-ended questions especially. just how, what a barrier that was to their performance and preparation even. So yeah, I think we've talked a lot about that as a faculty and just ways to mitigate that across the board.
00:17:37
Melanie Barlow
what do you think then, based on these results and studies and the quotes you've received, like, what does that say about our approach to you know, these practical
Role of Feedback in Learning
00:17:49
Melanie Barlow
assessments? Because I'm thinking... in the advent of AI and we need more secure assessments, simulation has to come to the fore because it's a secure form of assessment. So if people are looking to OSCEs more for that secure assessment, what would you recommend and like what does that mean for our students going forward with these high-risk assessments?
00:18:16
Kate
I guess there's always lots of considerations in that and I guess in the background we're thinking about staffing and funding and can we do that but are just reflecting back to an earlier comment of you know we need to make sure these students are practice ready for going on placement we know it's a secure assessment you could incorporate You know, if it was an individual assessment, you could incorporate more of an oral aspects and make sure students are talking through what they're doing and have criteria around that.
00:18:51
Kate
But I think because it's key knowledge that they have to know, I think feedback is essential. It can't be, know, I have a bit of an issue with the exams in that we're testing key knowledge, but then students don't get the feedback about the areas that they're and missing or that they need to strengthen
00:19:11
Kate
in that And it goes, it's the same sort of concept with a purely summative OSCE, if students are coming in and not getting the feedback and not understanding. Obviously, if things are really wrong, they'll ask be asked to come and do a re-sit and perhaps, and we know when students in midwifery anyway come back to do a re-sit at any point that they often nail it the second time. So that's also indicative of that feedback and the value of the feedback that students receive at that point to then come back and know what they're doing in that second instance.
00:19:51
Melanie Barlow
That's so true.
00:19:52
Lynnelle Moran
And I think these positive experiences, when that feedback, when positive feedback is modelled by the preceptors or by the teaching team, those students take that into their practice as well. And I think it's so generative and it's really important because so often it's punitive. You know, the hurdle task is pass or fail or, you know, feedback in the clinical setting, you know, once they graduate, it's, it's, very reactionary and punitive. No one has time to be preventative or look at, you know, areas that require development. And so I think it's a symptom of the current landscape and we can help students to develop those skills by modeling that through these assessment processes as well.
00:20:38
Lynnelle Moran
And they will take it into practice.
00:20:41
Melanie Barlow
Yeah, very good.
Advocating for Formative Assessments
00:20:43
Melanie Barlow
So I guess to wrap up, if people are listening to this and they want to implement it in their own areas, what's a bit of advice?
00:20:57
Kate
I think it's pretty easy to implement. So I think give it a go and see what, see, listen to the student feedback and listen to what they're saying, because I'm sure the students will really appreciate it.
00:21:09
Kate
And it's basically a repeat of the OSCE you've already got set up, but giving students the opportunity for a formative aspect of that and then a summative.
00:21:21
Kate
Obviously staffing's important and and those resources, So to build that capacity in to where you're going.
00:21:32
Melanie Barlow
Did you need to do any extra faculty development? Like what is the difference between a formative group, immersive simulation versus summative and OSCE and the role they play?
00:21:45
Kate
We didn't really, you definitely could do that. I think the team was pretty on board with, like we set that up well, like this is how it's structured and and this is the process, so set up the process really clearly.
00:22:01
Kate
Could definitely talk to staffing around formative assessment and I think the debrief in the middle and how to give that constructive feedback in the middle is important. I'm sure, Mel, you can talk to that. You're probably expert in that area more than I am, absolutely.
00:22:20
Kate
But that That's also a really important part to be able to give that really constructive feedback in the middle.
00:22:27
Melanie Barlow
Yeah, very good.
Conclusion and Personal Insights
00:22:29
Melanie Barlow
Excellent. so we ask all our guests this because we are Simulation Happy Hour. I have my coffee in front of me, which is regular. But if you could be and Happy Hour anywhere in the world, where would you be?
00:22:46
Melanie Barlow
And which what would you be drinking?
00:22:49
Lynnelle Moran
Oh.
00:22:50
Melanie Barlow
It could be anywhere from like, you know, in Italy with an espresso through to the beach with a
00:22:59
Lynnelle Moran
I love this question, Mal. I want to be in Iceland with a spiced rum.
00:23:06
Melanie Barlow
Nice. How
00:23:08
Kate
I've been really thinking about the Netherlands and Amsterdam lately. I love i love that place. So there it's probably cold there now. So Avam Chocolat, like a hot chocolate with cream on top.
00:23:23
Kate
It's very good.
00:23:25
Melanie Barlow
All right, take me to both those places. That's amazing.
00:23:29
Kate
Love the question.
00:23:31
Melanie Barlow
So Lynnelle and Kate, thank you so much for joining us today. And, yeah, I think that's a really practical activity that people can implement and, you know, create change straight away, it sounds like.
00:23:47
Kate
Thanks Mel, thanks for having us.
00:23:49
Lynnelle Moran
lovely, Esmel.
00:23:50
Melanie Barlow
Pleasure. Pleasure.