Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Episode 3: Assessment and Feedback
131 Plays28 days ago
This month our host, Stuart Norton, is joined by co-host Vic Stephenson and a wonderful panel of guests to talk about all things assessment and feedback. Two essential elements of higher education that all students encounter within their experience.
Our line-up of guests include:
- Professor Naomi Winstone, Director, Surrey Institute of Education, University of Surrey
- Dr Miri Firth, Senior Lecturer, Academic Lead for Assessments, The University of Manchester
- Dr Lauren Woodlands, Senior Lecturer, QUT
During the episode, guests provide thoughts on:
- Why good assessment and feedback is essential for student success
- The purpose of assessment and feedback
- Why flexibility and optionality are essential
- Key takeaways for those working on assessments and providing feedback
And of course we have our usual #EduBiscuit chat!
Find out more: https://advance-he.ac.uk/teaching-and-learning/lets-talk-about-student-success-podcast
Recommended
Transcript
Introduction to Student Success Pod
00:00:12
Speaker
Hello and welcome to our podcast Student Success Pod. I'm your host Stuart Norton and here we continue to discuss core thematic areas of student success. Please do remember to follow the series to make sure you stay up to date with new episodes and activity related to student success. I am once again joined by a co-host and in this series episode it is Vic Stevenson of Advanced HE with subject matter specialism in the area under the review today assessment and feedback.
00:00:42
Speaker
a topic that impacts globally.
Meet the Guests
00:00:44
Speaker
We have three fabulous guests from across the sector talking to all things assessment and feedback from their perspective and sharing their insights with you, our listeners. So I shall hand over to Vic to introduce herself and our guests. Hello, Vic. Hello, Stuart. Hello, everyone. and I am Vic Stevenson. As Stuart said, um I'm also a senior consultant in the education team for Advanced HG.
00:01:07
Speaker
And I work predominantly in learning and teaching development programs and offering consultancy and support around all things learning and teaching, but particularly around assessment and feedback, design and strategy. It's my great pleasure to introduce our three speakers today. And um I'm just going to have to take my glasses off. Apologies. So our first speaker today is ah Dr. Lauren Woodlands.
00:01:34
Speaker
and ah Lauren is a senior lecturer at Queensland University of Technology in Australia, working in the Central Learning and Teaching Unit. and Lauren's also the programme lead for a specialist Associate Fellow Award in Indigenous Knowledges for Practitioners in Australia, and this recognises successful integration of Australian Indigenous perspectives and knowledges within learning, teaching and assessment practice.
00:02:01
Speaker
Lauren delivers professional development programs, ah both for QUT and also for Advanced HE, working with me. um And her programs focus on inclusive assessment and feedback design, and the use of technology to enhance assessment from both a student and staff perspective. So hello to you, Lauren.
00:02:23
Speaker
Hi, everybody. Thank you for having me. Hello. I'm joining you today from Turable and Yagara land in Australia. That's the traditional owners of the lands on which I'm on. Thank you so much for having me. Brilliant. Thank you, Lauren. Our second guest is Dr Miriam Firth, who is the Academic Lead for Assessment at the University of Manchester in the UK. And Miriam led the QAA Optionality and Assessment Project and provides sector to support on flexible assessment. Mimi's spoken at a number of events, including the Advanced HE Assessment and Feedback Symposium panel discussion, where time was really too short for her to share all her insights on how we can enhance student experience and outcomes through the provision of choice and co-creation in assessment. And we're really looking forward to delving a bit further into these themes today.
00:03:17
Speaker
Thanks so much, Vic. It's such a pleasure to be here. I'm really looking forward to the conversation with Lauren and Naomi and yourself today. Thanks so much for the invite. Brilliant. Thank you, Mary. um And our final guest is Professor Naomi Winston, um who's Professor of Educational Psychology and Director of the Surrey Institute of Education at the University of Surrey in the UK. Naomi is also an honorary professor in the Centre for Research and Assessment and Digital Learning at Deakin University in Australia.
00:03:47
Speaker
And Nomi probably needs little introduction as she's been published or she has published over 100 articles, which are widely read and frequently cited, 6,433 in published works, according to Google Scholar. And Nomi's work focuses on learning behavior and engagement in us in education, and her research focuses on the processing and impact of instructional feedback.
00:04:14
Speaker
um When you Google No Meet, she actually comes up as a public figure. So we're really fortunate to have her join us today for our conversation on what makes a difference in students' experience and outcomes in assessment and feedback.
Biscuit Culture and Preferences
00:04:30
Speaker
Thank you, Vik, and thank you for the invitation. Really looking forward to the conversation today.
00:04:35
Speaker
So colleagues, welcome and thank you so much. As ever, the first thing we do yeah and by way of easing us into the episode is to start with something a little bit more hearted and that's of Edgy Biscuit. Listeners new to the series, you are in for a treat as our guests divulge if they are a dunker and provide their favourite biscuit for doing so. We are compiling our very non-academic chart to accompany the series, but as ever, this is a serious matter and to accompany my favourite research from the earlier two episodes.
00:05:05
Speaker
My fact this week is, or this month I should say, um the average Brit munches on three biscuits a day, which works out as 204 million a day across the UK's approximate 68 million population. Over the course of a year, an individual will consume 1,095 biscuits weighing an astonishing eight and a half kilograms.
00:05:29
Speaker
um But as ever, the academic side of this, Dr Hamid, a consultant in endocrinology and diabetes at Imperial College Healthcare and NHS Trust, said that bikies give us a dopamine high, which is even stronger if we dunk them. And I quote, dunking a biscuit softens up the texture and makes the biscuit literally melt in the mouth. So as these sugars hit the taste receptors and get into the bloodstream, that gives us that pleasure area faster.
00:05:55
Speaker
So with that very academic research supporting edgy biscuits, I'm going to come to Naomi first and ask you, are you a dunker? Absolutely not. I think it's horrendous to think about dunking a biscuit. It's ah a betrayal of their structural integrity. um And I think the dopamine comes whatever. It doesn't matter if it's been dunked or not. So no, I refuse to dunk a biscuit. Well, there you go. And what is your favourite biscuit, Naomi?
00:06:22
Speaker
It's quite boring, I think anything with with chocolate on, so chocolate digestive, chocolate hobnob. Classic. Okay. Can we push you for a favourite? Otherwise our chart's going to be, you know, all skewed. Hobnob. Hobnob. There we go. Okay. Okay. I think that actually puts the hobnob into first place from from from memory from earlier episodes. Miri, can I come to you next? Are you a dunker and what is your favourite biscuit? Absolutely. And the hobnob is now winning. The chocolate hobnob is the way forward. And it was so lovely to hear. I think it was your guest, Kate, on episode one.
00:06:53
Speaker
um referring to the chocolate hobnob. And I think it's an under undervalued biscuit across the biscuit market. I mean, it's just, it's solid, isn't it? It's just, it's just such ah an absolute delight. And unlike ah Naomi, I am absolutely a donker, but it has to be Yorkshire Tea as I'm from the right side of the Pennine. So yeah, it has to be Yorkshire Tea.
00:07:11
Speaker
um And I'm a multiple, a cereal dunker. Like I can't just dunk one. Like if I've got a packet there, I'm going for it. Fantastic. Helping push that average
Understanding Assessment and Feedback
00:07:21
Speaker
up to three a day. I like i like that, Miri. And having lived in the in in Yorkshire as well, we we still maintain Yorkshire tea in our cupboard. So that's, I i understand exactly where you're coming from there. Good to hear it. Lauren, ah please, are you a dunker and what is your favourite biscuit?
00:07:38
Speaker
So I'm not a traditional dunker, but there's a practice in Australia with Tim Tam biscuits that I know it's Tim Tams have come up a couple of times already. But what you do is called the Tim Tam slam and you actually bite the corner, the opposite corners off the rectangle and use it as a straw. You dip one side into the tea and you you suck the tea through. it And what happens is it makes the middle all soggy. You still get a little bit of structural integrity. but it's ah it's a very timed performance. You've got to instantly pop it in your mouth so that it kind of all melts together. And it's it's pretty spectacular, and you don't end up with any biscuit in the tea. um I occasionally dabble in this practice, but it's not my favorite biscuit. um But I just thought, I've got to let people know about that performance art that happened. Absolutely. yeah And um I do know, Stuart, you're still waiting for those concerns to be sent. um So listeners, um
00:08:34
Speaker
Please keep Stuart in mind. um But no, my favourite biscuit is the iced vovo. It's an Australian biscuit. And I want you to imagine the fanciest rectangle that you could get, like someone's giving you a picture frame and they've put raspberry jam in the middle and cushioned it either side with a bit of sort of fu ah ah pink fondant and then sprinkled it with coconut and it's an absolute delight it's it's particularly um fancy um and it just makes ah a cup of tea a lovely experience. Wow fantastic and a Tim Tam slam that brings me back to my school days when we used to use twixes as ah straws you know all these habits coming out. um
00:09:18
Speaker
Last but by no means least, Vic, um we've worked together for for for a long while now, and and I do not know if you're a dunker or not. Well, ah i um I would describe myself as a light dunker. i would I don't like any kind of biscuit getting into tea. I don't like dregs and things at the bottom of the tea. that is That's really not disgusting. I also am from Yorkshire, so it's Yorkshire tea only.
00:09:47
Speaker
and I would say a light dunking of a KitKat is something that really can't be beaten, where the chocolate melts just slightly but you don't get a soggy wafer. but I'm not a massive biscuit eater but when I do have them in the house it tends to be a KitKat, although I am not one of those people who can have two fingers or you know have half of it one I only ever buy those two finger ones have one and then leave half like it's the whole thing's going but I do snap them in half I think it's absolutely abhorrent when you see people eat a Kit Kat and they bite across the two fingers I've seen people do that before and they're shuddered. but That's just wrong Vic, that is just wrong. I really don't.
00:10:34
Speaker
Yeah, it's it's interesting, Vic. I almost feel like we're we're ah igniting a new a new controversy here. We've already had the Jaffa Cake in that in episode one. But is the KitKat a biscuit or a chocolate bar? And, you know, may maybe we put that to viewers to decide. It's wafer. It's only got a light covering. I know, but it's not biscuit, is it? It's wafer.
00:10:54
Speaker
um ah Okay. Yeah. I think, i well, I think, I think, you know, if that was an assessment, I think you might have failed there. Um, ah you know, we'd have to, uh, we'd have to check our rubrics and the marking guidelines.
00:11:07
Speaker
Well, with that in mind, and, uh, thank you colleagues for ah sharing share in those, uh, intimate biscuit moments. Um, without any further ado, I think it's time to talk about student success.
00:11:18
Speaker
So today we are talking about assessment and feedback. And first things first, what what is the first thing that comes to your mind when you hear the term assessment? I'm going to start with you, Miri, and we will come to all of our guests in turn. So Miri, when we talk about assessment, what's the first thing that comes to mind when you hear that term? Thanks so much, Stuart. So I think for me, the first thing that comes to mind is opportunity.
00:11:43
Speaker
um It's really a chance where i can sit I can find out where one of my learners is at, you know how far they've come, um what their potential holds and in in a lot of ways. I think it's like a snapshot and it it tells it tells me a story of where they've got to, you know whether that's about their growth from the start of the year, any gaps in knowledge or new directions that I need to support them.
00:12:09
Speaker
and But ultimately, I always think assessments about an opportunity. It's about finding out something um new about my learners and supporting them through their next stages. Yeah, no, absolutely. um it's It's the one area, isn't it, that that that that reaches every single student without foul across the institution, across any program. and And so it is so essential and integral to that student learning journey and the experiences that students have at university. Naomi, can I come to you next? Yeah, I think the first thing that that comes to mind is is a word like responsibility. It's an incredible responsibility on us as as educators, ah researchers and as as institutions and um building on what Miri's just said, I'm reminded of um
00:13:01
Speaker
quite a well-known statement from David Baud, which is it's it's easy for students to escape the effects of bad teaching, it's impossible for them to escape the effects of bad assessment. So if we get this wrong, then we're actually going to have a ah really significant impact on our students. And I think one of the reasons why um we don't always do the best we can in terms of our responsibilities because there's often a lot of conflict about the purpose of assessment. um And is this a way of certifying learning? Is assessment a way of checking learning? Is it a way for learning to happen and skill development to happen in the first place? And yet we often forget, I think, one of the most important purposes of assessment, which is feedback for us as educators to be able to see in our students' work where they're at, what impact our teaching has had. and
00:13:48
Speaker
what it is we might need to to do differently in the future. So it's a huge responsibility, but also I think there's a lot of conflict around the purposes of assessment that really need to be unpacked. Yeah, it's a really interesting point. And as you were talking, it was reminding me of standards. and You know, we use assessment, don't we, to to assess the the the standards of where our students are. But it's not quite as simple as ah as you know so standards of a bicycle safety helmet, which meets the particular standards. And we know that we have confidence in that. There there are so many different variables in there as well. And and yeah just thoughts around calibration, which I'm sure will come up at some point later. Anyway, um Lauren, can I come to you next, please? Yeah. um The first thing I think of when I think of assessment is that's where the learning happens.
00:14:37
Speaker
um We always think that it's in the classroom when we're telling students things, but it's actually when they're off completing assessment that they're really having that experience of learning. um And I feel that there's often a missed opportunity when we create assessment to create experience for learners because it's often this finished task that students are focusing on rather than really understanding all of that.
00:15:03
Speaker
gorgeous opportunity for metacognition, learning about themselves, how work happens, how questions are formed, how you can wonder about things. um And so I always encourage people to think, okay, what's the assessment piece, but what are all the opportunities that we can create for learners around that with learning? Yeah, thanks, Lauren. No, that's ah that's a really valid point. And that reflection piece ties in perhaps with a ah follow-up question and we'll we'll go back in reverse order and around feedback which which you know absolutely essential and and combined when we talk about assessment in general we we typically mean assessment and feedback but when we we when using the word feedback what what does what does that mean? Miri I can see you've got your I can see you'd like to come in there and please do.
00:15:54
Speaker
um I think, I mean, firstly, I think it's so fascinating, the the first answer to what assessment is, because I i think in just to sum that up before we go on to feedback, I think it's so interesting the levels of assessment literacy our students have, because I think we spend so much time looking at this, thinking about it, discussing it with our accreditors. And yet there's so much more work we can do with our learners. And that's the same with feedback. You know, how many times do we go around our classrooms offering that formative feedback verbally and informally?
00:16:24
Speaker
And yet, you know, if if we went round with a big sign over our heads going, this is feedback, you know, it would suddenly make so much more sense to our learners. And I think the thing for me about feedback is that it's a dialogue. And I see that with our assessment as well. You know, good feedback should be a dialogue and should be a continuation. Shouldn't be the end point. Shouldn't be fixed.
Challenges in Feedback Delivery
00:16:45
Speaker
Yes, it should refer to any intended learning outcomes.
00:16:49
Speaker
But ultimately it's about celebrating the progress and then helping that learner to see the path forwards. Even at the end, you know, when we've done PhDs, second PhDs, et cetera, there is always more to be improved upon, considered um and learnt forward. So I think dialogue is the key thing for me around um feedback, but I just wanted to pick that thing up around assessment in the classroom because I think assessment literacy is something that we're working really hard on here at Manchester and it's just so important and we can't all walk around with these you know these big placards saying this is feedback now this is you know and they see the assessment and feedback's really hard to underline all the time and when it's always there.
00:17:33
Speaker
Yeah, thanks, Mary. um I've been with Advanced HE now. It'll be my ninth year this year. And prior to that, 13 years in academia. So as as a recovering academic, you brought back memories of my my days in the classroom when I did actually have a placard that said feedback. and And I used to hold it up and show it during during my seminars um because I thought it it was something that was really interesting that students only saw feedback at that point of the the formal assessment. um And so, so yeah, I just did bring back memories from a past life there.
00:18:02
Speaker
happy ah nao me I'll go back round as we did before. Yeah, thank you. This is a really difficult question for me to answer just to say the first thing that comes into my mind when I hear the term feedback, because I spend most of my day thinking about feedback and what it is and what it means. um If I were to pick something, I think I would say for me, ah the most important thing that we certainly need to be thinking about with feedback is the idea of of impact and change.
00:18:30
Speaker
So the idea that that that feedback is not the words that we might write or type or even speak to our students in response to their their work. That's just information. It's comments. It's an input into a process. And the real magic of feedback and and the power that it has to enhance learning and motivation and skill development comes from what happens next. And that what happens next is is driven by students. So for me, the most important thing we need to be thinking about in terms of feedback is not how we deliver it, not how we write it, what kind of feedback form we use, what kind of language we use, um certainly not using things like the feedback sandwich, um those little techniques that supposedly enhance learning. What we should be doing is um really thinking about how we empower students to to make use of feedback and actually see that change and impact in in the short and the long term. Yeah, it's really, really
00:19:23
Speaker
valid point there. Sorry, vix we going to come I was just interested in the because I know that, you know, I've heard ah about the, you know, I've heard about it, obviously a lot. the The feedback sandwich was actually cited in the in my PG cert as kind of good practice. And I'm wondering whether that is still something that is kind of because it's not something I would advocate, as as you said. um But what Why would you say, what do you think the the real problem is with with feedback sandwiches? It's just that students know the i think you know they they know that it's kind of, you're you're buttering them up for kind of, you know and it's quite inauthentic. You're not asking them for their perspectives first. What what exactly is your issue with feedback sandwiches?
00:20:11
Speaker
and Yeah, so I've tried to move beyond just having a personal issue with feedback sandwiches um as being like just the simplistic technique that we can use to actually try and to develop some evidence for for why it's problematic. So um together with ah a colleague, ah Anastasia Lipnovich at City University of New York, we've been doing a series of experimental studies trying to look at what impact the feedback sandwich actually has on what students do. um And in these studies, what we found is is that when the the really important ah critical developmental advice and feedback is preceded by this sort of general statement of praise, it actually leads to students making less improvements and less motivation to improve their work. And we describe this in terms of an effect called the anchoring bias, um which is quite a ah common cognitive bias that some initial information, sort of anchors or positions are perceptions
00:21:04
Speaker
in a particular way which will then influence how we will process subsequent information. So that initial statement of praise anchors students judgments and saying oh this is okay I'm all right I'm doing fine. They just hear what they want to hear. sort still spring up Yeah and then it means that when they come to process the critique they perhaps don't take on board as much of it as they would do without the praise. um So hopefully we'll we'll see more of these these studies, replications of these studies coming out and suggesting that It's nice to have little strategies that we can use in feedback to to take off some of our cognitive load when we're thinking about how to provide feedback information. But actually, we do need the evidence to be able to say, is this going to do more harm than good? Yeah, yeah, absolutely. Agree. Thank you. Lauren, I'll hand over to you.
00:21:54
Speaker
Yeah, i want to I want to build on something that Naomi touched on when she's talking about helping build students resilience and ability to engage with feedback and that's I guess thinking about how as educators we can understand the role that emotions play in learning and how feedback is something that a lot of people have already had many different experiences with so by the time you know they come to us in our classroom it's it's such a it's already a loaded feeling attached to it um
00:22:31
Speaker
One thing that I try and get educators to do when I'm when i'm helping getting getting them think about teaching is do an empathy map on what does it feel like to get feedback. So you think, what is it what does it make you think? What does it make you feel? What does it make you say and what does it make you do? And if they start unpacking their own experience and feelings, it helps them um gain a little bit more empathy into the process of engaging with feedback as a learner. so And that also helps you think, like, how are you coming to giving feedback? Are you trying to justify a grade? Are you trying to um rationalize a particular approach someone should or shouldn't have taken in their work? And so if you unpack your feelings, it can help you think about, well, what is the message I'm actually trying to give? And how am I building this person up? And how am I helping them engage with this feedback in the first place?
00:23:23
Speaker
um So, you know, we we have such an important role in the way in which we we craft and manage the feedback process with students.
Assessment Literacy and Choices
00:23:32
Speaker
Yeah, I think that's a really, really interesting point you made there. And and and in the earlier question around the first thing that comes to mind, I also, as Mary had had put, as an educator, assessment provides opportunity, an opportunity for you to demonstrate, an opportunity for you to learn and to to enhance and and develop. But I've noted that as a student,
00:23:53
Speaker
The first thing I thought of was fear, because I have a distinct recollection of walking to two my finals at university and actually thinking, if I got hit by a car just now, I could like get out. I was so terrified. And as you're saying, putting yourself in that position of why was I so afraid? And part it was because it was all, and you know, high stakes, high security, long ah written exams.
00:24:21
Speaker
But there was also, you know, there was no, at that time in the nineties, it wasn't something that we, you know, that, that your lecturers talk to you about, about how to prepare and what you were doing it for and what happens next. And so I think it's really useful, as you were saying, to think about your, how you felt in assessment situations and in feedback and how it can vary. I i tend to not focus on the good and focus on one, somebody said one comment that was, that was not less than perfect. And that becomes my focus. So it's, it's really interesting trying to take yourself, you know, put yourself in your own mind, but also take yourself out of it because everybody responds so differently. It can be a real challenge to, to kind of, yeah, to make sure that everybody feels supportive and benefits from getting good assessment and feedback. Sorry, Miri.
00:25:19
Speaker
I just absolutely agree with what you've just said. I cannot do exams at all. Like I'd love to be able to do exams, but but I think that's the thing for me. I always reflect on my own, you know, assessment experiences. And I think what you've just said just so resident it resonates with me, Vic. Likewise, I just, I just panicked because I knew that when I'd done exams previously,
00:25:43
Speaker
you know, my ability in that stressful situation to recall that information just wasn't as good as it could be. You know, I could dance it, I could sing it, I could act it, but get me in a room where I've got to be quiet with a pen and I just couldn't, you know, and I just wouldn't perform in inverted commas as as best as I could. um And I think that's why, you know, and while we're discussing feedback, actually giving feedback aligned to learning outcomes is a massive, ah is of massive benefit as well, because I do think that while students will be, you know, my learners will be listening out for where the praise and the critique is coming from. I always try and grab the unit handbook and say things like, well, let's have a look at this learning outcome. Do you think you've met this? And if so, how? So instead of it always being our feedback and our colleagues' feedback, it's about getting them to self-assess, getting them to self-reflect and also um allowing them to to think through
00:26:40
Speaker
what that learning outcome actually means and how they are performing it within that assessment time. um But yeah, and I'll talk later about the optionality aspect, but I think optionality in assessment is absolutely the way forward to enable us to avoid those negative emotions, because it gives that autonomy, it gives that control, it it greatly motivates students um to actually succeed better. Yeah, absolutely. Naomi, I can see you've got your hand up next.
00:27:07
Speaker
Yeah, I was just going to add a ah reflection on the the conversation here because I think it's really interesting. What we're talking about is the importance of humanizing assessment and feedback and humanizing our role as as educators in in that. And this reflection on our own experiences, remembering what it's like to be on the receiving end of assessment and feedback, I think is is is really important. but more important to me is that next step of then actually sharing that with our students and being really open about our own struggles with assessment and feedback. And it reminds me of what um Margaret Berman and and Liz Malloy write about as intellectual candor or ah as they had originally turned it, intellectual streaking, which is the idea that as educators, there's real real value to us exposing our our weaknesses and our vulnerabilities to our students and telling them that we struggle too. and
00:27:56
Speaker
I do this with feedback. I will often share with students examples of really challenging feedback that I've received, whether that's teaching evaluations, appraisals, um peer review comments, and will be really open about the anger that they might raise in me or my feeling that I just can't look at them today. They're going to have to go away and and I'll have to come back to them a few days later. And sometimes the students will say,
00:28:21
Speaker
ah Are we allowed to put feedback to one side and come back to it later? And I always find that a really interesting conversation that they have this sense they need to engage with it straight away. So I think it's huge power that we have if we can show some of that vulnerability and be willing to be be um really candid with our students about our struggles, really help to normalise the process and give them confidence that they're not getting something wrong. Dealing with feedback is ah is ah a challenge for humans. It's not a student problem, and something we all face.
00:28:52
Speaker
Yeah, absolutely. And it can it can be really challenging for students to to to see that the the feedback is on the work, not on them personally. And I think in certain disciplines, particularly things like the creative arts or performance arts, that's really, really challenging where students have put their heart and soul into something and it's very personal. And any feedback, you know, needs to be managed really carefully. And as Miri was saying, you know, with reference to kind of criteria and and and learning outcomes rather than kind of, oh, you could have, been you know, and um when when you reflect on, I don't think I ever saw a criteria when I was at university. I have no clue, you know, it was in your in your academic heads, you know, you didn't have that conversation. Nobody showed you that. So I think it's it's so valuable that that becomes part of our that is embedded in our practice now. Sorry, Lauren.
00:29:45
Speaker
Well, good. i I was just thinking about my own experience of keeping chapters of my PhD with lots of track changes that I had gotten back from supervisors. And it was this wall of red where it was almost indistinguishable from my original writing and the the changes that had been suggested. And I i keep it so that when I um have one-on-one consultations with PhD students,
00:30:14
Speaker
but I can show them and and it's especially valuable for non-English speaking international students where English isn't there their home language and I can show them and say i'mment I'm a ah native English speaker and this is the level of feedback that I'm getting with this sense of normalising that someone's engaging with your work, they're they're wanting to ah tell you a little bit more or or change your thinking and get you to understand, it but normalising that that engagement is a really good thing that um someone's, you know, someone's looked at your work and it's um it's actually, a you know, a gift in a way. um But I always also refer back to there's a fabulous bit of research called the Higher Education Learning Framework. And one of the key things from there is the fact that emotions play such a big role in learning and it can either
00:31:10
Speaker
inhibit learning or it can promote learning so it's how can we get on board with the emotions that are going to promote learning and so that normalizing that we're doing of engaging with feedback that is one of those things that is just so vital to helping build resilience in learning. Yeah there's some brilliant work where there was but another QAA collaborative project on compassionate feedback and there's some really useful practical prompts for you to kind of think about considering in the way of delivering and it does cover things, you know, encouraging students to think about their own state of mind and their own their own ah emotions and and timing and whether it's the right time for them to receive this sort of feedback. um Absolutely. So we've kind of talked a lot around kind of, we've we've established that, you know, high stakes, you know, exams
00:32:04
Speaker
are terrifying for many necessary in some in some cases, we wouldn't want to say nobody should ever do exams. I did hear a case sample was saying though, if you're, if you're setting a timed assessment exam, under exam conditions, you have to have a really clear reason for doing so. And it has to replicate the sort of situation a student would find themselves in in in the real world, rather than it's because that's when you can book an exam hall for that amount of time and, and you know, youre really not considering the student. But I'd like to come to Miri where, when we're talking about assessment and it meaning lots of different things, what you mentioned before about obviously your work on optionality in assessment, how do you personally define what good assessment looks like?
00:32:58
Speaker
I think it has to involve some element of of student choice um and I know that through presenting on our optionality project in in various conferences and and obviously we've got publications through the QAA website as well I know that by me saying that I'm automatically raising you know people's concerns and and fears around um you know what that might mean so so when we do refer to optionality and assessment I do not mean allow every student to do whatever they want first and foremost. um Our national project looked at I think it was 13 different forms of options in assessment and I think good assessment embeds and is built on student choice by allowing students some form of flexibility um and as you've mentioned i'm I'm the academic lead in our flexible learning program so my my focus is very much around
00:33:56
Speaker
embedding flexible learning at the University of Manchester and supporting that through student choice in assessment. And one of the most straightforward areas that you can offer choice in assessment is around the deadline. um So when colleagues are saying to me, you know, but how can I do this, you know, we've got a number of markers, I've got a cohort of 700. If you think about when you're deadline for whatever assessment it might mean, that assessment is going to be uploaded or created into your VLE and it's going to be opened at some point. From the moment that that assessment is open on that on that online platform to the moment it's due, there is choice there. That student or learner in your cohort can submit it the day it's open and available to submit up until that minute, an hour, and day that it closes. And I just think sometimes being absolutely explicit with our learners to say, you know,
00:34:51
Speaker
you need to do an essay for this piece of work or it's an exam but up until that point you have this time within which to submit that look at your schedules think about your time management you know would it benefit you to actually submit this a week early would it benefit you to submit on the date because again offering that um option that autonomy it immediately motivates learners and that's what our research study ah found time and time again. you know and We surveyed hundreds of students, hundreds of staff, we did workshops across the board and it was the thing that we found was most beneficial to our students. Don't just give them lots of choice, identify where choice is available and that will automatically make them feel
00:35:39
Speaker
more in control of that assessment and more autonomous to to really be more creative ultimately and shine. So yeah, I really think some form of choice is essential. That's brilliant. Thank you. And if if anyone hasn't seen Miri's post on LinkedIn, which is ah like a cartoon strip or comic strip rather, really setting out what optionality and assessment can mean. Because I think it's quite it's quite threatening in some ways for for for some academics to think. you know you Like you said, it doesn't mean you've got to allow them to submit a poem or a 2,000 word essay. you know there are There are different levels and different ways in which you can approach optionality and flexibility. And um working out what's right for your programme and discipline m is is obviously key. I'll hand over back to Naomi or poor Lauren with the same sort of question. but What do you think
00:36:35
Speaker
um really defines a good assessment.
00:36:42
Speaker
um I think good assessment is something which it goes beyond subject matter. I think a lot of the time assessment is just solely focused on the the content that we're we're trying to get students to to demonstrate that they know. But when you think holistically about that whole human being that we get to develop when we're helping our our learners become, you know, learning is ah is a sense of the coming. So why focus just on this one piece of it and think about, well, what are the skills? What are the attributes? What are the qualities? What are the graduate attributes that we're really trying to help build in learners? And whether or not that's something that isn't always demonstrated in in the rubric
00:37:31
Speaker
which is something which is you know often such a ah small space that we can and really look and assess, is it something that they can develop in in the process and the way they think about learning in the conversations that you have with them? So that is the key for me, is does the assessment look beyond just the subject matter itself and think holistically about the person that we're that we're trying to develop?
00:37:56
Speaker
and know Yeah, I think and in assessment design decisions, we're often pushed to be driven by by pragmatism um and often the the real power in assessment comes from thinking in a lot of detail about what's most meaningful for for students in that situation. And at the moment, obviously, we're in a situation in in the sector here in the UK as as well as elsewhere where where efficiency is a word that we're talking about a lot. And I think there's a risk that that pushes us again towards thinking pragmatically about assessment.
00:38:27
Speaker
How can we save time? How can we make things simpler? But I would argue if we've got precious resource that assessment is where it needs to be directed to make sure that we're designing tasks that are going to enable students to really develop and learn, not just test their understanding. We shouldn't be driven by thinking about how easy is this to mark. We should be driven by Is this the right assessment at this time in the student's journey? And is it the thing that's going to actually ah lead them to enjoy what it is that they're doing and working towards that task rather than it being a chore?
AI in Assessment and Feedback
00:39:00
Speaker
Absolutely. I think we just um we've focused a lot haven't we on on the human element of of assessment and I know Stuart was keen to pick up on on something you mentioned there around ah the pragmatism and especially in the age of AI. the drug the I think it's it's coming back the other way, but there was ah an initial
00:39:18
Speaker
shift towards let's get students back in exams and Stuart was going to pick up the theme of of technology there. Yeah i I think it's an interesting point isn't it because there was definitely that shift back to the exam hall because then it was we know this is the student's work and you know rather than debating the rights and wrongs around that it was something else you mentioned there as well Naomi which was how easy is this to mark and and and I think there there is sometimes a challenge around large cohorts, you know, how do you get through three-week turnarounds, that that real pressure for for metrics, NSS, you know, everything, all of this is under scrutiny, even though we know that that certainly from a yeah UK perspective, you know, that assessment feedback is the area that students typically respond, you know, less positively on. But how is technology reshaping assessment and feedback? And that may be in terms of of positives, it may be
00:40:11
Speaker
be in terms of challenges, it may be in terms of the unknown, but but what works and and and what what should we be leaving behind? Yeah, I'm happy to um give some thoughts on that one, Stuart. I think, for me, the the the general rule of thumb that we need to keep in mind when we're thinking about the role of technology in assessment and feedback is that we don't just jump on technology to recreate less than optimal practices using a different medium. um And if we take feedback, for example, there's a huge, I suppose, what some people see as an opportunity right now with generative AI to say that we could outsource the generation of feedback comments to to generative AI, free up a lot of academic time, speed up the return of feedback. And on one side, that's a really positive set of of outcomes that that that we could see.
00:41:01
Speaker
However, what we're doing there is merely replicating the delivery of comments is quite outdated representation of feedback using a different medium. um And I would argue very strongly for the importance of keeping humans in that in that process. We know students, we understand work, we understand what's gone into producing that work in a way that artificial intelligence never never can. So what if we could say, yes, generative AI could produce comments, the input, the bit that we're least interested in in the feedback process. And that frees up staff time to do some of the things that really empower students to use feedback, to have those conversations, to support them to set goals, to track progress towards goals, all of the things that really make feedback impactful. um We could see generative AI as as a positive in that sense.
00:41:51
Speaker
so What I think this illustrates is is not um being seduced by the idea of technology saving us time or doing something um that really merely replicates the delivery of comments. We need to use technology purposefully to think about what can this do to enable us to enact the kinds of processes we really want to see happening in higher education. um And that I think shifts the mindset towards thinking about affordances rather than getting trapped in um this idea that technology is going to solve all of our problems and assessment feedback. I think that's so, so important Naomi. and And I think just on the back of what you've said of the potential benefits of AI, where we did a study with GISC two years ago, um trying to understand students' perspectives of f AI use and assessment. And we found universally that students are wanting to be led by us. And I think if if we as educators are
00:42:49
Speaker
supporting the use you know to brainstorm ideas, ah you know to refine ideas, to even you know check understandings from different perspectives even. I just think it it can have that potential for a really positive and ah positive tool for for assessment. And ultimately, it has to be a new skill, doesn't it? you know We cannot have lifelong learners of the future that have been told you know through any education program. They're not allowed to use any form of AI because it's just, it's unrealistic, isn't it? you know When we were talking about you know exams having to be authentic and appropriate, that's exactly the same as technology. If it's appropriate and relevant, go for it. If you're just excited by a new piece of kit, maybe don't use it. It's those aspects, isn't it? It has to be done appropriately and and authentically. authentically
00:43:42
Speaker
um I think AI gives a wonderful opportunity for giving students a space to be able to ask vulnerable questions during their learning. Like if we think about maybe creating a specific space where students can go in and ask particular questions about um what steps they should take or whether or not they're on the right track with their learning or what um what could they do next or how how they could improve on what they've already started.
00:44:11
Speaker
um This is a really wonderful way to um provide a um opportunity for students to really start thinking about what they need in learning. And there's also some wonderful research that talks about the importance of then getting students to reflect back on how they've used that AI as part of the learning process. um And that's where they can really consolidate that learning as they're developing those skills for lifelong learning in an AI reality.
00:44:42
Speaker
Thanks, Lauren. It's really interesting there. And, you know, immediately when we talk about technology, there is that huge focus on on on AI. and And I think, undoubtedly, because the you the rise of chat GBT and many other AI bots, I mean, who can believe it was, well, just just over two years ago now that they came into being in and the huge exponential growth and their ability to to it to expand as well.
00:45:08
Speaker
I just wondered if anyone had any experience of other examples of technology and then around an assessment. and and And again, I'm just thinking around um you know the the growth of virtual reality. um you know We have seen in in certain disciplines simulations,
00:45:25
Speaker
um you know sort of through medicine, through to aviation, and also in in applied criminology, specifically policing, that ability to to engage in 360 videos, that virtual reality world.
00:45:37
Speaker
um I don't wish to put any colleagues on the spot, but it was just something that came to me as we were talking around different use of technologies. Does anyone have any comments or experience in that regard? Naomi, can I come to you? Yeah, I think I've seen some really fantastic examples of this, sorry, right from quite typical uses of simulation, I suppose, in healthcare and health professions education.
00:46:05
Speaker
but through to some quite novel ah uses of of ah digital tools that enable quite realistic simulation and and facilitate group work as well. um In our hospitality and tourism management programs, for example, students do some fantastic simulation work running hotels and and and making really quite complex decisions with data. um Really well received by students, feels meaningful, supports employability,
00:46:31
Speaker
um enable students to connect with each other around a meaningful task. I think the challenge with these kinds of tools is is that people might feel they don't have the expertise to to develop, to use and to implement them. And that's why I think central teams can be really, really useful in in helping um to connect people from different disciplines who might be wanting to develop something similar or or might be facing a similar challenge within their their assessment design and and trying to to connect those people and and and um facilitate the development of novel uses of technology and assessment.
00:47:07
Speaker
Yeah thank you it's really interesting and I don't want to diverge the of conversation as well but it's almost like gamification around learning as well isn't it in that virtual reality world and I know you know gamification is It's also something that that has expanded greatly as as an avid board game player. I'm delighted when I see posts on LinkedIn and and where else talking about the benefits ah of of playing games. Do we think that students' views and approaches around assessment have changed since this increase in the use of AI?
00:47:41
Speaker
um Vic, what are you saying from ah from from from your sector position? Do you think that's something that's changed? I mean, I think it's it's really interesting. I think there's a lot of assumptions made around students' use of AI and their desire to use it and their desire to use it to cheat the system and and all of that. I think um my experience in actually some of the research that came out of our collaborative development fund with a number of universities in Scotland um showed that students are are generally
00:48:12
Speaker
um very uneasy and very unsure about, I mean, there might be proficient users of AI, but where it comes to so using it around in an assessment context, it it throws up lots of feelings of, if I don't want to use it in case I get caught and I do the wrong thing and I don't mean to do the wrong thing, but then if other people are using it and I'm not using it, then they might get a better mark than me and I'll be disadvantaged. So I think it's really It's interesting that um I've seen certain ah institutions take a very kind of strong line of kind of we must stop it, which I think is impossible, but also making assumptions about how students feel and what students want to do. And actually, exactly as Miri said, they need guidance from us on and from there from their lecturers on wander how they can use it responsibly and what that means, because there's a lot of policy around
00:49:12
Speaker
ethical and responsible use. But for a lot of students, that what does that mean, ethical use of AI? What what are they supposed to do with that? And you know the research showed that and there's still an alarming number of students, as sorry, lecturers, who are not talking to students about AI.
00:49:31
Speaker
at all and I think that's, you know, that's really a critical first step. Sorry, Mary, you wanted to... No, it's really fascinating because that map matches so much with the work we did with GISC and that original report to the sector on student opinions and the thing that we found which is also often um not explicitly referred to is is how the lecturers are using it. You know, one of the things we found was that the students were were thinking you know, we're now using this, we have the opportunity, what are my lecturers doing with this, you know, and and we heard a resounding concern over any use of AI for marking, any use of AI for lecture content. um And I think it's again, you know, just as you've said, Vik, it's about saying to our learners, this is how you can use AI for this particular assessment. This is how I have used AI in these certain ways, you know, whether it's
00:50:27
Speaker
prompting Menti to create a discussion question or whether it's, I don't know, there's all sorts of things that it can do, creating a scenario and then and then checking it afterwards. But I think it's that, it's the dialogue, it's the frankness, it's the transparency.
00:50:42
Speaker
you know, I'm a full advocate of being open and um honest about the use of technology where it's appropriate, because what you'll get back is then that student feedback, which we need, you know, not just end of term and not just for NSS, but the students will then go, well, why are you using it like that? and And then it enables that dialogue, doesn't it? and But it is both ways. It's not just us confirming how we want learners to use it. It's also identifying how we might have um used it for our teaching.
00:51:11
Speaker
And i'm I'm sure colleagues on the call are aware, was it the word delve that suddenly jumped up in academic articles within the first 12 months of chat GPT being used? you know we don't We don't even have to talk about it. you know It's obvious it's been used. I just think it's far more human to be that person in the room to say, yeah, actually, I couldn't think of a scenario last night when I was planning for this you know this discussion today. So I asked chat GPT for an example of a scenario. That triggered me to then think further. you know Be open and honest about it.
00:51:41
Speaker
um Anything you'd like to add, Lauren, from from the other side of of the world? Presumably students are students, you know, they they've got the same sort of, you know, fears and concerns and ah and ways of working. But ah is there anything do you think that's potentially a bit different? Because I noticed that you, I've i've referred quite a lot to the Texas principles and and that they're really, you know, it felt to me like Australia was very forward thinking was very proactive as to how can you build this into your assessment? Where is it appropriate? What guidance should you give? And there seem to be some really clear guidance right from the start. And obviously the work that came out of the University of Sydney um is brilliant. The the the student authored site on yet good tools to use, how to how to use prompts effectively, how it can help you in your discipline.
00:52:38
Speaker
you know, it's just brilliant, brilliant guidance. um And so much work has gone into it. So it'd be interesting to hear your perspective from Australia. um Yeah, I do also highly echo the there's some fantastic stuff by Texas talking a lot about, um you know, assessment reform in the age of artificial intelligence.
00:53:01
Speaker
um And I think one of the the main things that comes out of it is it's not just one disciplined body that says this is the way of approaching it and doing it, but the fact that there's multiple approaches that you can take and different ways of, um you know, considering um how you want to use it in your teaching, how you want your students to use it in learning. And kind of like what you said, it's just being really explicit with how does this unit incorporate AOI How does my teaching incorporate it? How should your learning incorporate it? And modeling you know ah what the expectations are in the same way that we would expect there to be that degree of explicitness in a workplace scenario with you know what can we do ourselves?
Trends in Assessment
00:53:49
Speaker
What can we outsource? What can we bring in artificial intelligence um for as well?
00:53:54
Speaker
um but More and more the the comments that we get from students are just really echoing that they they're looking for guidance. What am I supposed to do? How am I supposed to do it? And how can I do it well? ah Just just conscious of our listeners from across our global our global reach. And and maybe I'm feeling slightly overconfident there in that we're we're having hundreds of thousands of you listening in. But Texas is the tertiary education quality and standards agency in Australia.
00:54:25
Speaker
who are the independent national quality assurance and regulatory agency for higher education. So I just wanted to cover that one off. I'm just just aware of and inclusivity around terms and acronyms, um so something that I'm being much more conscious of myself these days. Vic, sorry, i know I know you're going to lead on to trends.
00:54:46
Speaker
Yes, no. So I think we've we've we've kind of picked up on kind of what what's happening in different areas. And we're interested to hear if there's there's anything that is that you're seeing in your own institution, any trends or themes that is that's impacting your approach to assessment and feedback within your own particular context. Mary. Yes, certainly. and And again, this must come with the caveat that I'm coming from the flexible learning ah program at the University of Manchester. So looking at assessment from that lens, certainly we're seeing lots of trends. and I think
00:55:23
Speaker
I'll refer to five briefly just around some of the things that we've done around optionality. um So that there are more diverse assessment practices, as the first one. So students want assessments that do cater to their unique strengths and learning styles. And there is a growing interest in that in terms of non-traditional assessment formats. So things like collaborative projects, portfolios, presentations. Now, I know they they won't be non-traditional to lots of colleagues, particularly and in different institutions, but at a Russell group like ours, they are usually more non-traditional. And our students want more of those, want more access to those across the board, um irrespective of discipline. Secondly, um more inclusive and equitable assessment, so seeking assessment methods that account for the diverse backgrounds and reduced bias. and We've got an excellent team in in our library through My Learning Essentials, so Jenny Blake and the team,
00:56:21
Speaker
look at lots of flexibility for assessment to support inclusive practice, those students with disabilities, those balancing work and study um and also those underrepresented groups. So we've got lots of activities happening around that to support those learners. yeah And that one most notably because flexible learning is now the norm you know across the sector.
00:56:44
Speaker
It's not just about mature students coming back in and requiring part-time study that's still in a 9-5. We've got learners globally that want to dip in and dip out transfer credits. So I think that inclusive and quite an equitable assessment has jumped. It's it's grown from and from just looking at diverse and under-representative groups to being those range of flexible learners. um I think the integration of formative feedback. so Students really value our feedback when it's timely, actionable and developmental. So we've got much more stricture stricter requirements around the formative feedback when it happens, um discussing that with peers, using peers in class for that as well. um Number four, technological integration. So having things like e-portfolios, online submission platforms, things that are more integrated, more agile.
00:57:39
Speaker
um that support more innovative assessment types, again, like I said before, for more authentic assessment, you know, just because we use VLEs doesn't mean we have to submit assessment that way, you know, just because it has worked in that way last year doesn't mean it works, you know, this year. um And finally, I think the key thing, and again, this helps back to our research on optionality, it's around enabling agency and co-creation.
00:58:06
Speaker
Students not just being presented with a fait accompli that is, we need you to do these 6,000 words by this date within these learning outcomes, but having a dialogue through assessment, enabling them to think about the design of their work, how they best want to produce information on their knowledge and their development that showcases that best. You know, who is better to do that than the individual? Why should it be an academic that sets that rigid structure?
00:58:35
Speaker
when it can be done in dialogue and it can support that agency and co-creation. So there's five things there that i'd I've thought about before our discussion today, but I suppose that's just the starting point within our flexible learning assessment at Manchester. And I know that they'll they'll be echoed with colleagues. Yeah. Thank you, Miri. Naomi? Yeah, I think there's some really, really important um themes there in terms of what Miri's talked about.
00:59:01
Speaker
I think for us, the the the trend, the focus that we've had has been on zooming out from assessment as just being the thing that happens when a student has submitted a piece of work and we we grade it and provide feedback information. We've been thinking about how we support student learning in preparation for that assessment as they're working towards it, but also how we really support their learning after that work has been returned to them and how we help them work with feedback and and and help them to apply it. And the the contextual ah challenge that I think we've been aware of in doing this is often when we're trying to do institutional change in assessment and feedback and any other areas of education it can feel really overwhelming for um staff that they've got to make a big change they've got to do something really dramatic whereas actually what's often really useful is is lots of small changes um and so we've used this approach based on the aggregation marginal gains
00:59:57
Speaker
Um, that's today, Brailsford used when, um, trying to improve the performance of team GP, uh, team GB cycling team. Um, the idea that if you kind of take part riding a bike into all of its little components and try and improve each one by 1% when you put all that back together again, you've got a really significant improvement. And we've done the same thing with assessment and feedback said, if we break assessment feedback down into all of its component parts and try and make tiny tweaks that improve each of those.
01:00:25
Speaker
we put that together and it does add up to a significant change. So focusing on assessment of feedback as a full cycle, not just the the process of of grading and providing feedback and doing that through the aggregation and marginal gains rather than trying to um make big shifts that ultimately are and are not sustainable within practice. Brilliant. and Thank you, Naomi. And Lauren, you've got the unenviable task of of kind of adding to to what has been too really comprehensive.
01:00:54
Speaker
i'll well that look i i'll I'll go the unusual route and I'll go short and sweet. the Something that I think has really come up in the Australian sector a bit, particularly at my university, is this idea of trauma sensitive, trauma informed, trauma aware education, and the idea of how how that can sit within the assessment and feedback process in in how we can prioritise the relationship that we build with learners around learning and around thinking about learning. um you know So that's something I'm really excited to see prioritised within the sector. Great. Thank you, Lauren. i'm So if we can move on to to your top tips, which I think Stuart has got as a feature throughout the the podcast series,
01:01:43
Speaker
So if I could ask each of you to think of what what is one top tip or tangible takeaway that you would like our listeners ah to try out going going forward. Nomi, shall we come to you first? Yeah, I think one of the things I feel really strongly about is is that we need to disentangle, we need to separate assessment and feedback. We always talk about assessment and feedback together as if they always have to happen together. And actually, I think that's where we we as a sector have got ourselves into and so some really challenging situations in how we make both assessment and feedback effective. And so my my key tip or takeaway is to actually think about separating feedback in a design sense from assessment. We always talk about assessment design, we very rarely talk about feedback design, whereas what we actually should be doing is thinking where in a cycle, a learning cycle is feedback and the time that we can spend on feedback going to have the maximum impact
01:02:39
Speaker
In very few cases, is that going to be right at the end after we've graded a ah summative task? So separating out feedback, designing feedback for learning, thinking about where can we embed feedback so that it can actually have an impact, rather than feeling we've got to be shackled to assessment and feedback always happening together, I think is is my key takeaway. Great. Thank you. Yeah, I couldn't agree more. And I think you're absolutely right. It is assessment and feedback in one, in one mouthful, isn't it? And it and it really doesn't.
01:03:08
Speaker
need to be and it shouldn't be. Miri, can you come to you? You're going to guess this. It has to do with options. I'm going to go there. It really is just so powerful. um So yeah, if there's one thing I think I'd really like your listeners to do moving forwards is to introduce any small element of choice in either the assessment or feedback and it doesn't have to be both.
01:03:36
Speaker
um And it certainly does not need to be a complete overhaul, you know, so start with something manageable. um So for assessment, um it might be something like I've mentioned already, where you outline to those learners, you know, your deadline is on the 12th of June, but you can you can submit up until that, you know, there is choice in that, you know, look at your time management, think about when you need to submit and when's best for you.
01:04:01
Speaker
um You can pilot things and leading up to that assessment as well, you know, a lot of our colleagues will do some drafting, you know, you could have options in that, so it doesn't have to be an option in the assessment, which is the final submitted piece either. It can be, you know, we've been learning about this topic, I just want to check on where your knowledge is up to in week six, so in that class you've got an option, would you like to write me a few words on it? Would you like me, would you like to, you know, rap about it? You know, so then you're removing those high stakes as well, and you're just making it a more enjoyable, um more free flowing and flexible um process of building up to that final submission. And in terms of feedback, um you know, you can also allow them to think about the type of feedback they'd like.
01:04:53
Speaker
you know, we spend hours, hours and hours and hours. I mean, I, I'm really quite OCD about this. I love standardizing. I love looking at the marks per question or marks per intended learning outcome. I'm really thinking about that year after year. That's me. I'm not saying everyone should do that, but you know, having that open discussion with learners to say, I am planning to give you written feedback. It normally takes me you know, this amount of time. But actually, if it's more beneficial for you, for me to and talk that through and give you an audio piece of feedback, let me see if that's going to be possible. And again, make it a dialogue, you know, don't shy away from um opening up that dialogue and and suggesting different things where it's feasible. You know, I don't suggest you, so you know, you open up the conversation on options unless it is feasible and checking with your colleagues.
01:05:45
Speaker
But letting students choose that feedback format again is incredibly powerful. And also, um you know we've did talked today around when we feel comfortable in reading feedback.
01:05:58
Speaker
you know, do our learners want feedback on every intended learning outcome or do they want more depth on their writing style or their knowledge on one particular learning outcome because they've already reflected and already identified a gap in their ability. So therefore, are they wanting us to think about that as well? um So yeah, definitely think about a bit more choice. Don't change the entire system. Do check with colleagues. Think about what's feasible.
01:06:26
Speaker
present it to your learners and say, look, I can offer you a bit of choice here. Would you like that? Would it be beneficial? If so, let's go with it. But give me feedback on that as we as we continue. Yeah, I think that' that's really beneficial in getting students to take some responsibility as well of their own, what they what they how they get their feedback and what they do with it.
Infusing Compassion in Education
01:06:49
Speaker
um I think the point that you just mentioned there that just occurred is um of I've seen comments before and that we tend to focus very much on standardization of assessment. But there's very we rarely any kind of standardization activity that goes on around feedback. So within any given department, you can have some lecturers offering choice, some offering am having it in a very dialogic approach, some where it's written comments, some where it's very brief, some where the tone of the comments is quite negative. and So I think
01:07:25
Speaker
it's It's kind of worth exploring as well, obviously not within the remit of this podcast, the you know strategies and ways in which standardization of or feedback and um what's expected and what options um academics have. Because for some of them, you know it can be they can be quite new two ah to marking and providing feedback. And it feels like there's very little, there's a lot of training around how to mark, but less so about how to give feedback.
01:07:55
Speaker
and Anyway, Lauren, your top tip. um my My top tip is ah is a practical activity that you can do with learners and this is around getting them to actually create a bit of a feedback map um and they can um map out all the different places, people, sources that they can get feedback from on their learning. And the idea being that at different points during the the course of the program or semester that you're running, you can get them to revisit that map and go, well, what have you been mainly relying on? And which do you see as more important or reliable? Or I see here that you've been using a lot of written feedback from your tutor, but what about going to a writing circle with your peers? Or how about
01:08:37
Speaker
um have consulting with a librarian or looking at the literature, like what are some different ways that you can engage with feedback? And this is really to open up learners, I guess, understanding of how there are so many different ways and formats and sources that they can get get feedback on their learning. Thank you, Lauren. Do you know, I love that idea of a feedback map, um you know, this idea of seeing all those different sources and actually them having that as a visual map.
01:09:05
Speaker
I think there's ah there's an idea of ah of a blog there for Advanced HE to share some of those ideas, Lauren. um Colleagues, thank you so much for your time, Stan. I do want to have like one last question, and and it's really around, is there a call to action we should be putting out to the sector? um What areas should providers be looking to work together? you know where Where is that one space? And I'm conscious that that your own research takes you um across the globe and already, but is there is there anything in particular um that that's so that that we should be doing and and that call to action. And Lauren, I'm going to come to you first because I've seen you just put something in in the chat to us there and it speaks volumes to me. So so please. Oh, yeah, I was going to say that the biggest, the one word that we can throw out is compassion, compassionate feedback, compassionate assessment, compassionate learning. Because if we if we approach anything with compassion, we're bringing it back to the human. Yeah, human absolutely. And I think compassionate dialogue as well is just so so valuable and something that's missing and and also a sector that's under pressure. It's something that that we've practiced. We can be more compassionate to to our to our colleagues and and and and ourselves as well.
Conclusion and Resources
01:10:15
Speaker
Naomi and then Mary.
01:10:18
Speaker
Yeah, I completely agree with Lauren. I think it particularly with the conversations that are going on about Church EBT at the moment, and and artificial intelligence, looking at how we we we really maintain humanity within within assessment and feedback processes. But more importantly, I think um in answer to your question, it's not so much what collaborative providers should focus on in collaboration but how we do it, we see so much sharing of what works, um we very rarely see sharing of what hasn't worked and actually I think for people who are leading institutional change in assessment feedback, knowing what other people have tried that hasn't been successful is actually way more useful because then you don't replicate those ah in your own institution so I think more willingness to share things that haven't worked would be really really valuable.
01:11:08
Speaker
Yeah, that's so true Naomi and anyone who knows me knows that I always bang the drum for what doesn't work as well. And it is absolutely critically important. And yeah, very much aligned to, to you know, when you were talking about ah marginal gains as well, all of these areas I've looked at previously and and really do speak volumes. So thank you.
01:11:26
Speaker
Miri, one area for you, that that that point of collaboration, what what should providers be looking to work together more on? Yeah, thanks so much. You're you're doing it already. And what we are. And I think that's the thing, keep having the conversation. I mean, exactly as as as colleagues on the call have said, I just think having those corridor conversations are so important. You know, either if it's after you've moderated something or after you've presented the assessment task and go, I'm not sure how that landed. i'm you know And just having those those conversations is so important with colleagues. And you know building capacity and time and culture to enable that will just automatically improve assessments. So I just wanted to add that to Naomi's comment on on that collaboration and firstly. But I also think
01:12:13
Speaker
Whilst we've developed the resources with the QA um on their website around optionality and assessment, I think there's actually a call and perhaps Advanced HE will be interested in doing this with us, but looking to create a flexible assessment framework for the sector I think is definitely needed. We need to collaborate on.
01:12:31
Speaker
what works and what doesn't work and and I fully agree with with talking about the issues and the barriers that are presented because some of the feedback we had was around the discipline peculiarities around supporting flexible assessment You know, and it's it's not to do with discipline culture, it's more to do with the creditors. So where you have a particular creditor who only supports this form of assessment potentially or or not others. So I do think there's a call for us to look at developing a flexible assessment framework. And I'd be delighted to work on colleagues with that if they're interested. Thank you, Mary. And and I'm sure there will there will definitely be interest across the sector and certainly from sector bodies.
01:13:11
Speaker
um as As you're talking there, it's reminded me how all of the core thematic areas of student success are are interlinked. you know we've We've touched upon graduate outcomes, we've touched upon flexible learning. and you know we We don't have time to to to get to go into the impact on on different groups of students and international students and assessment around there as well. um and And there was just so much more. and And I'm sitting here thinking around a podcast, around collaboration, i'm sorry, and around ah standardization,
01:13:38
Speaker
and calibration, um which is which is something as well because we talk around assessment, but then it's that market of assessment and calibrating across the sector as well and and and into that. So that leads me to say thank you to all of our guests, Mary, Lauren and Naomi, for joining us today, to Vic, my co-host on this episode. All of your insights were absolutely invaluable.
01:14:02
Speaker
Before we go, I wanted to remind our listeners that there are a wealth of resources, including the recent literature review conducted by colleagues at the University of Kent, Dr. Ed Pitt, and Professor Kathleen M. Quinlan. And indeed, their gamification of the research, assessment and feedback superchargers, a freely available resource available to all Advance HE members. We have the assessment and feedback framework, and there is a forthcoming symposium in late 2025. All details are available on our website.
01:14:30
Speaker
And myself and Vicks will be co-editing another series of assessment and feedback case studies, which we published later this year. We're back next time with a podcast in flexible learning, and much more can begin to be unpacked with what promises to be an equally exciting episode. Until then, it leaves me to say that biscuits should be sent in to Advanced HE, Kerry Junaughton. But stay curious and do tell your colleagues and friends, be sure to hit that like button and follow.
01:15:24
Speaker
Thank you and goodbye.