Transcript
Speaker: Hello and welcome to our latest episode of our podcast.
Speaker: My name is Chris McNutt and I'm part of the progressive education nonprofit Human Restoration Project.
Speaker: Before we get started, I want to let you know that this is brought to you by our supporters, three of whom are Peter Kratz, Janet Clark and Corinne Greenblatt.
Speaker: Thank you for your ongoing support.
Speaker: You can learn more about the Human Restoration Project on our website, humanrestorationproject.org or find us on social media and YouTube.
Speaker: Today, we are joined by Angela Stockman.
Speaker: Angela is a veteran secondary English and language arts teacher, author, and professional learning facilitator.
Speaker: She has presented at state, national, and international levels and has led curriculum, assessment, and instructional design projects in over 100 different school districts.
Speaker: She has written books and resources on writing instruction, including the Writing Workshop Teacher's Guide to Multimodal Composition, Creating Inclusive Writing Environments in the K-12 Classroom,
Speaker: in the recently released The Writing Teacher's Guide to Pedagogical Documentation, Rethinking How We Assess Learners and Learning, which we're talking about today.
Speaker: So thank you, Angela, for joining us on the program.
Speaker: Oh, I feel so incredibly privileged to be here.
Speaker: Thanks, Chris.
Speaker: Yeah, let's just dive into it.
Speaker: Let's kick some things off.
Speaker: Let's talk about your most recent book, The Writing Teacher's Guide to Pedagogical Documentation.
Speaker: And it has this really kick-ass hook in the book description.
Speaker: I was just checking out on Rutledge right before we jumped in.
Speaker: And it says, this book is a call to action for English and English language arts teachers who understand that data are not just numbers alone.
Speaker: learning is impossible to quantify and students are our very best teachers.
Speaker: So before we get into the specifics of the book, could you talk a little bit about what it means to document learning if it's impossible to quantify?
Speaker: Because I feel like it's very different than the norm.
Speaker: It is.
Speaker: And this has been quite a journey for me.
Speaker: I was excited to write this book because this is, I'm coming up on my 33rd year in the field.
Speaker: And from the moment I walked into my first classroom,
Speaker: I was very excited to teach writing, and it's been a huge passion of mine that sustained me throughout my career.
Speaker: I feel very fortunate to be the age that I am and still just as excited, if not more so, as I was when I was a brand new teacher that very first year.
Speaker: But yeah, historically, my experiences with assessment have had everything to do with
Speaker: measuring performance toward an outcome.
Speaker: There's been a heavy emphasis on summative assessments, more common forms that look like multiple choice or essay writing that we bring to a rubric in order to produce a grade.
Speaker: And that grade becomes the ultimate indicator of student success or failure.
Speaker: And the more that I've learned
Speaker: the more I've become interested in far more humane and human forms of assessment, which include pedagogical documentation.
Speaker: Pedagogical documentation is really about making learning visible.
Speaker: And a lot of my work around that and my learning there has been inspired by Project Zero, Harvard's Project Zero, and Reggio-inspired educators who have understood this for a very long time, that assessment is a verb.
Speaker: It's something that we do.
Speaker: It's not something that we give.
Speaker: And when we make the learning process visible and we use all of these fabulous technology tools that we have at our disposal now to take pictures or capture audio or video or even just sit in conversation, quick conversations too, with learners, and they have the ability to make their thinking visible
Speaker: using representative materials.
Speaker: So for me, a lot of loose parts play, a lot of drawing, a lot of opportunity to talk or perform and not just reducing everything to written words and written numbers.
Speaker: We're actually able to learn so much more about students' strengths and their true needs.
Speaker: It makes us better diagnosticians too.
Speaker: So I find that when we rely on the numbers, um,
Speaker: it doesn't really tell me enough about what students truly are capable of.
Speaker: Um, and then I end up making the wrong assumptions often.
Speaker: And so documentation kind of allows us to make the full catastrophe of learning visible.
Speaker: Um, and we can scoop that information that we need, that we want to hold onto so that we can be far better, you know, assessors and more reflective.
Speaker: It makes that possible.
Speaker: And, um,
Speaker: For me, that is the most exciting and rewarding part of living during this time when we have such access to technology, including AI.
Speaker: AI gives us great potential there, too.
Speaker: More about in a second, because I'd love to ask a follow-up surrounding AI, because I wonder about some of the, I guess, the pushback that might happen.
Speaker: Let's just talk about that for a second.
Speaker: She's probably valid.
Speaker: I mean, what's interesting about AI is I was recently at a, well, I was at it with you.
Speaker: I was at NCTE.
Speaker: And one of the things that stood out to me at NCTE on the conference floor is the growth of AI grading tools, AI feedback tools.
Speaker: They're kind of taking the education world by storm.
Speaker: And what really stood out to me was that
Speaker: they're not just like nice to have, oh, the AI is going to help me write a comment on the student's work.
Speaker: It is very much like, here's the rubric, A, B, C, D, hit a bunch of check marks.
Speaker: Is that the right grade?
Speaker: Yes or no.
Speaker: And it's meant to speed up or make it more efficient.
Speaker: Yeah, we can make bad stuff more efficient.
Speaker: Yeah.
Speaker: So like, what does it look like to use AI in a more humane way?
Speaker: Because I feel like in educational, that's not the case, at least not right now.
Speaker: So I can give you an example in that three weeks ago, I brought K-12 teachers together over the course of five days.
Speaker: We were broken out into grade level cohorts.
Speaker: So K-2, 3, 5, 6, 12.
Speaker: And they brought to the table documentation that they gathered in their classroom in the form of pictures of students' work, audio recordings, transcripts.
Speaker: They brought all sorts of student work samples that were
Speaker: where kids were using loose parts to represent the parts of a story and then speaking over it in ways that had them using really high-level vocabulary words.
Speaker: And one of the biggest findings that came out of our work was that when students were using loose parts to represent their writing ideas, they were using high-level vocabulary words that they might not have initially had the print power for, but they were able to
Speaker: translate that like they were willing to hang in and include those words and metaphor and symbol in their writing because we took it one small step at a time.
Speaker: But the reason why we were able to draw these conclusions is I was using an app that captured 36 hours of audio and then pulled out the highlights, summarized our conversations so that I could put them back in front of everyone efficiently.
Speaker: It didn't take me because it usually has in the past two weeks.
Speaker: at home, some are all in that sort of data.
Speaker: I had it immediately.
Speaker: And I did code the data by hand.
Speaker: And what the AI tool did was absolutely in alignment with my own human coding work.
Speaker: And so to be able to instead of, and I truly believe that historically, we've reduced things to numbers that have diminished our understanding of what students know and are able to do
Speaker: because it's efficient and it's all we've been able to do.
Speaker: But while I'm not so much about plugging a rubric in into a tool and using it to generate comments on student work, and I'm not ever about putting student work into an AI tool without their consent to generate feedback, although it does a good job of that, I am all about tools that will help me capture on nearly 40 hours worth of audio.
Speaker: human conversations and it was sitting in my pocket.
Speaker: It was non-invasive.
Speaker: It allowed me to be more present.
Speaker: And those conversations could be a little bit meandering, which was necessary.
Speaker: And it produced conclusions rapidly that allowed us to really discover things that we wouldn't have otherwise.
Speaker: And that's the kind of AI I'm here.
Speaker: Yeah.
Speaker: I mean, I'm, I'm certainly a pro technology person too.
Speaker: Uh, I think a lot about the SAMR model, uh, where the last level of the SAMR model was, is redefinition.
Speaker: And something that's interesting about what you're talking about is that essentially that's something that you really couldn't do in the past.
Speaker: It would not be feasible for an average teacher to listen to 36 hours of audio recording, um, and actually make something of that.
Speaker: Um, however, do the growth in large language learning models, uh,
Speaker: You can.
Speaker: And that's really an impressive feat.
Speaker: And it's really cool.
Speaker: I can't help but self-plug.
Speaker: That's also how we do our PD.
Speaker: We essentially are recording hours upon hours of student voice work and then running it through an AI model to help get our themes.
Speaker: Obviously, there's a human component.
Speaker: We're not just like tossing it out there and just hoping that it works.
Speaker: But it would be you would need a team of hundreds of people to look to these results to do it any other way.
Speaker: And you were there for, like, this is what I love too.
Speaker: And I'm sure it's your experience as well.
Speaker: I was present for those conversations.
Speaker: So when I'm looking at what the AI is generating in terms of conclusions, and I give it back to the team too, we're still in that whole experience.
Speaker: We're impacting it.
Speaker: We're evaluating it.
Speaker: We're deeply engaged with it all.
Speaker: And so we know whether or not it's doing what it needed to be doing.
Speaker: Yeah, yeah.
Speaker: Going back to just like traditional, I guess, pedagogical documentation, steering it back to that part of the conversation.
Speaker: Something that you were saying reminded me of when I was teaching.
Speaker: It was early in my teaching career, like year three, year four.
Speaker: And I remember I used to give a lot of multiple choice tests because it was very efficient.
Speaker: And I was a history teacher at the time.
Speaker: It was like 10 question multiple choice tests every Friday.
Speaker: And I remember I used to have these students who I thought were doing very well in class, seemed to always participate, always knew the right answers, etc.
Speaker: And
Speaker: they would get like a five out of 10 on these tests.
Speaker: And it didn't occur to me sadly until year three or year four to ask them, okay, why are we not getting them?
Speaker: So I remember I pulled these students aside and they were questioned surrounding, I think the civil rights movement.
Speaker: So I'd ask them, you know, like, oh, well, like you put here, the Martin Luther King Jr. thought this, or that Malcolm X thought this, et cetera.
Speaker: And then they would be able to explain the entirely right answer to me, even though that's not what they marked on the multiple choice test.
Speaker: And that led to this conundrum where, first off, I had a little bit of an existential crisis because I thought about all the kids I probably gave Ds or Fs to, thinking that I was doing the right thing, but they probably knew the right answer.
Speaker: But also, it led to some introspective thinking where we could figure out different ways of assessment.
Speaker: So what are the other ways that we could capture those moments that you write about a lot in the work?
Speaker: we landed on portfolios, but I'm interested to learn more about the examples that you share.
Speaker: So there's everything.
Speaker: There's like the photo voice method, there's a documentation notebook.
Speaker: What does it actually look like to capture student learning in this way?
Speaker: So I think from my position inside of this work, I'm usually a facilitator who's brought in to work with people who are coming from very different places.
Speaker: I have very inexperienced teachers.
Speaker: I have some veteran teachers.
Speaker: I work in very different
Speaker: parts of the world.
Speaker: And so the first thing that I think of is comfort level and how brittle the teacher is that I'm working with.
Speaker: And brittleness is usually the effect of being completely overwhelmed in their work.
Speaker: And so when I'm working with a teacher who's basically walking in the door going, girlfriend, I have no time for this.
Speaker: I need to be very realistic about
Speaker: What sort of documentation approach is really going to help this teacher gain traction and really fall in love with the process because it makes that teacher feel far more effective at what they're doing.
Speaker: And so early on, it might be something as simple as documenting a moment.
Speaker: And my friend Aaron Shorn and Will Rupin at Unruler talk about it in that particular way, like,
Speaker: If there's something very specific you want to learn in a lesson, maybe you're only going to document one moment in that lesson.
Speaker: Maybe you're going to document your direct instruction.
Speaker: Maybe you're going to document conversation that you have with a student, one student even.
Speaker: So it can be as simple as documenting what a student says, what a student does, or what a learner produces in a learning moment, or yourself, what you said, what you did.
Speaker: what you produced in a moment by taking a picture.
Speaker: And I do this all week long where I will take a ton of pictures and I'll wait until the weekend or when I have a free moment, I literally have next week where I'm going to be sitting with probably over a thousand photos I took over the last month because I haven't had time.
Speaker: And then start to interpret them.
Speaker: Documentation can also look like a journey where maybe you're going to document how you're teaching and
Speaker: inside of a writing mini lesson over the course of a week or a month or even an entire school year.
Speaker: When I work with teachers around improving performance, they might have identified in one district that I'm working in right now, it's become very evident.
Speaker: We have a lot of evidence that's suggesting that regardless of the grade level, students are struggling to both identify and express main ideas in their writing.
Speaker: That's pretty significant because the ability to do that impacts a lot of things as readers and writers and learning human beings.
Speaker: So we've made that a focus for the entire year and we're not documenting everything.
Speaker: We're documenting how we're teaching that concept.
Speaker: We're documenting how students are thinking about it and we're documenting what they produce and we're bringing that data back to the table.
Speaker: And so I think it's really important to have a focus
Speaker: to understand that it can be something micro, like just a moment, and you're going to take pictures or capture one video.
Speaker: But then usually what starts to happen is you become comfortable.
Speaker: It becomes sort of innate in your practice and not overwhelming.
Speaker: And it's deeply rewarding to do that.
Speaker: Portfolios, you know, are beautiful.
Speaker: I'm excited because I was just invited to serve as an assessment facilitator on the New York State Plan Pilot team.
Speaker: New York State is piloting with dozens of school districts the option to provide students the opportunity to pursue portfolio performance-based, project-based pathways instead of in lieu of the regents exams.
Speaker: So I'm super excited to be a part of that pilot.
Speaker: Portfolios can be big, right?
Speaker: And that can also be something that students are polishing up and having to do a lot of revision work around.
Speaker: Documentation is more about the process.
Speaker: And you can have a process-based portfolio as well.
Speaker: And I honestly think that that's a lot of what emerges at the end of a documentation project.
Speaker: I had my students in my assessment methods course at Damon University document their learning for the entire semester.
Speaker: And at the end, what they basically have is a portfolio.
Speaker: But then I think of my daughter, Laura, who's a designer and her partner, Zach, he's an industrial designer and they have these gorgeous portfolios that they have posted online so that potential clients can uprope their work.
Speaker: So portfolio has some different sorts of translations, but yeah, it can be something very small.
Speaker: And for me, when I'm documenting my learning,
Speaker: I'm often moving at the speed of light and I want to be really present for my students.
Speaker: So I'm just capturing things and I'm not stopping in the moment, you know, necessarily to reflect deeply, but I'm taking a picture, I'm grabbing a video, I'm grabbing student work, knowing that when I sit down to look at and interpret it, it's going to be deeply meaningful.
Speaker: Yeah.
Speaker: And it sounds like too, in a lot of the situations that you name in the book, students are the ones that are selecting the
Speaker: which forms of documentation best represent their learning.
Speaker: So in many cases, it's the student who's taking the pictures or reflecting on the pictures that are provided.
Speaker: And is there a meta component to that?
Speaker: So I wonder if one of the things, for example, that English teachers are looking to do is having students identify main ideas in their writing.
Speaker: that also means that they would be identifying main ideas and they're learning through the documentation process.
Speaker: Yes.
Speaker: So how do we get like more meta with that?
Speaker: What are the skills that we're learning by doing pedagogy in this way?
Speaker: Well, I think that what's really beautiful, and it's something that's hard to translate for teachers that are not involved in these conversations often.
Speaker: The beautiful thing about pedagogical documentation is it relies on multimodal expression.
Speaker: So the thinking is that
Speaker: We're not just going to rely on alphabetic text and print numbers and letter grades to tell us what students know and are able to do.
Speaker: We're going to invite them to perform or draw or build something.
Speaker: And that enables invites them to use different modes of expression.
Speaker: It requires us to use different modes of documentation.
Speaker: And it also relies on conceptual transfer.
Speaker: And so when we start talking about going meta with all of this and trying to explain to students what, here's what a main idea might be.
Speaker: Do you notice it in a story?
Speaker: Do you notice it in this photo?
Speaker: Do you notice it in this video?
Speaker: Do you notice it in this animated short that you just watched?
Speaker: Now let's look at your work.
Speaker: What's the main idea of your work?
Speaker: And so what's gorgeous is that
Speaker: More than any other assessment method I've ever used, pedagogical documentation really facilitates another level of transfer, an added layer, and that can be integrated into instructional design in ways that I feel is very sort of organic, where other forms of assessment, the stuff that we've traditionally used, like multiple choice and essay, you're kind of bringing the learning to a halt.
Speaker: in order to test whether or not it happened.
Speaker: And then you're getting this lousy data typically that requires you to do more inquiry anyway.
Speaker: For me, documentation is a beautifully integrative approach to assessment because it too is multimodal in nature and it really plays well with teaching for transfer.
Speaker: it's one of those things to me that just feels more authentic and real in the quote unquote real world, which the school is the real world, but school is one of the very few places where you're ever going to consistently take quizzes and tests or a document, the things that, you know, you might take one of those at the end of like college, like you might have your bar exam or something, but,
Speaker: For the most part, it's not common.
Speaker: Or if it does happen, it's used for the most basic things that no one cares about.
Speaker: Like I think about safe schools for teachers where it's like, what's the right chemical thing to use on this type of fire?
Speaker: I don't know.
Speaker: There's all these strange multiple choice tests that we have to take.
Speaker: But beyond that, you are doing multimodal expression.
Speaker: You are having folks come in and observe the things that you're doing to tell a story about how you do your job.
Speaker: Or people might, I mean, actually take pictures and videos of your work and use that in order to do improvement plans, et cetera.
Speaker: It just seems like it would prepare students for not only, it's going to prepare students not only to learn more, but just to have more robust learning experiences down the road because they're used to it.
Speaker: They're going to understand.
Speaker: Yeah.
Speaker: What I love.
Speaker: So I use a lot of different tools for documentation, depending on the context that I'm in.
Speaker: But one of the tools that I use is Unruler, Aaron Shorn.
Speaker: And I know that, that, you know, Aaron and probably.
Speaker: And so Aaron, one of the beautiful things that I love about Unruler is it looks a lot like Instagram.
Speaker: And when I'm working with my students in Unruler, they start to, and documentation does this as well, they start to realize that learning moments are just as worth remembering as maybe that trip I took with my friends last week.
Speaker: And it sort of starts to unschool the learning experience and turn it into something that's worth building memories around.
Speaker: And that can be incredibly empowering and rewarding for students, particularly those who might have sort of been beaten up with grades in the past, right?
Speaker: And they start to realize Maggie Burns, who is my former student, she's an intern now for Aaron and Will, focused deeply on documenting her learning, illuminated a really bad relationship that she had with perfectionism.
Speaker: And she kept documenting her learning after my class was over because she wanted to, because it was helping her learn things about herself.
Speaker: I think what all of us who are connected online and who have been connected educators through social media have been doing that has brought us into relationship with each other is we've been documenting our learning and sharing it on Twitter and LinkedIn and Instagram.
Speaker: So that is a really beautiful thing.
Speaker: And I really think it starts to shift how we value learning and where we value learning outside of the walls of school.
Speaker: Testing, all of those tests, really, we give them because teachers want to have certainty and leaders want to have certainty that some basic things have been learned.
Speaker: And that's okay.
Speaker: There's a time and place for multiple choice.
Speaker: I'm not going to totally diss it.
Speaker: There are times when I will use selected response because I just want to know, you know, do they know the basics?
Speaker: But yeah, I think documentation helps me do so much more in students too.
Speaker: Yeah, it gets to the heart of something that I've always been obsessed with, which is I'm sure that you find when you change the assessment systems in a class, it has the additional effect of changing the entire pedagogical framework of what a teacher is doing.
Speaker: Because at the end of the day, if I'm a more...
Speaker: traditional legacy, whatever word you want to use to describe a teacher who does more of like the PowerPoint multiple choice test question format.
Speaker: That's not a very exciting thing to document because all I'm really documenting is some notes.
Speaker: And then the next day I'm documenting more notes.
Speaker: Yeah.
Speaker: And you can document and I have how items perform, can unwrap test items and get at content, context, cognitive load, those sorts of things.
Speaker: And I do that sometimes too, but
Speaker: That's really about analyzing how items perform.
Speaker: That's not really documenting significant shifts in our thinking, our learning, our work, or the culture.
Speaker: And that's where the good stuff is.
Speaker: Yeah.
Speaker: So then do you find as folks work through the workshop that they start thinking to themselves, well, I know kids are going to be documenting all these cool things that they're doing.
Speaker: That means I need to make more cool experiences like project-based learning in order to make that happen.
Speaker: Like, does it lead to the, yeah.
Speaker: Yeah, I think sometimes what's interesting is I think sometimes there's a perspective around beginning work with pedagogical documentation.
Speaker: And I'm often thinking about this as a facilitator.
Speaker: Sometimes there's a perspective that, you know, I walk in a door and we start documenting our learning and all of a sudden everybody, you know, there's fireworks and confetti and everyone's just living in this new peaceable kingdom that's so ultra creative.
Speaker: That is not at all what it's like.
Speaker: I often find that students and teachers alike, there's some scaffolding that goes on into that work because it's very uncomfortable.
Speaker: And yeah, one of the conversations I have to have often is, what does it feel like when we're learning?
Speaker: And I know that we have these, and I said this last week in sessions that I was in, we have these very cliched conversations about learning is frustrating and
Speaker: And failure is a part of it.
Speaker: And you want to fail fast.
Speaker: And we've been having that conversation for like 20 years now in the field.
Speaker: I'm really talking about a human being, an individual, understanding the embodied experience of learning that is very unique to them.
Speaker: And I think that we get disconnected from that very rapidly.
Speaker: And unless you understand that feeling for yourself, you don't know when to document your learning.
Speaker: unless you're being really directed to do it at a certain moment.
Speaker: And so in the beginning, it does feel like teachers are selecting and students are selecting very specific moments to pause, reflect, and document.
Speaker: But where you want to get eventually is to a place where it is intuitive because I am self-aware and I know when I am learning and what that feels like for me.
Speaker: And that is changing how I learn.
Speaker: And when that starts happening, then you start to see a shift in teachers' relationships with students and students' ownership and influence over the curriculum and also how things are being taught.
Speaker: That takes time.
Speaker: It doesn't happen in the first two weeks, right?
Speaker: It is definitely a process.
Speaker: I have made that mistake.
Speaker: I remember when I first shifted over to portfolios, I got really into the idea of unschooling the classroom.
Speaker: And the very first thing I did was like, well, I'll just put, you know, 20 standards up and kids will just tell me whenever they learn about those standards.
Speaker: And that's it.
Speaker: It'll be great.
Speaker: It'll work out fantastic.
Speaker: And I learned right away that does not work because first off kids freaked out because like
Speaker: How am I supposed to know which one of these things goes in which standard?
Speaker: It's like you're ripping the rug out from under them.
Speaker: Someone has always that's always been the teacher's job.
Speaker: They haven't paid attention to it.
Speaker: They haven't been developing that facility in any way.
Speaker: Right.
Speaker: I also didn't have any resources prepared to help them.
Speaker: Like I didn't anticipate how much work it would take to have that scaffolded journey.
Speaker: Yes.
Speaker: You know, by the time I left the classroom, that was an ongoing process that happened at least once a week.
Speaker: We did portfolios.
Speaker: But we did not start there.
Speaker: It very much started like where you're saying, where it's like, hey, on Friday, we're going to talk about what we learned this week.
Speaker: And here's like one standard.
Speaker: Which one do you think that goes with?
Speaker: And it was very much like an elementary look at how do we choose the things that we're learning.
Speaker: But it does get to that point where kids, like one of my most proud moments concerning portfolios is when kids would do something in another class and then come up to me and say, can I use this thing I did in my fine class for your standard?
Speaker: I was like,
Speaker: yeah, that's amazing.
Speaker: And then a bunch of kids would replicate it.
Speaker: And then you just have like a lot of people.
Speaker: Yeah.
Speaker: And then it starts to roll.
Speaker: That's where you really start to get traction that moves you toward down a new bend in the whole path.
Speaker: That's really exciting.
Speaker: Conversations that I sometimes have with teachers and administrators too, where they'll, there will start to be a lot of frustration expressed around standardized testing.
Speaker: You know, we spend all of this time preparing for the tests and executing the tests and grading the tests.
Speaker: And we just want to do project-based learning and we want to use portfolios because the time alone.
Speaker: And I'm like, okay, let's take a step back here because if time is really the factor we're playing with here, you may not be satisfied because actually this approach takes just as much, if not more time,
Speaker: to execute well and to make meaningful.
Speaker: And I think it, but what I find is it just keeps us all teachers too engaged in the work and rewarded by the work.
Speaker: And if you can be intellectually curious and not take it personally, when things don't go well, you know, when you put the 20 standards up and the kids freak out, you didn't quit and just say, well, this isn't going to work.
Speaker: No, you kind of leaned into it like, all right, how am I going to, it becomes a fun problem to solve in some ways too.
Speaker: And that's really the spirit that you have to come at all of this with in order for it to be rewarding.
Speaker: Yeah, it helps you connect with people too.
Speaker: Like that to me was the big standout moment was that I think I was always pretty good at having like positive relationships with kids, but not necessarily connecting with them at the learner level.
Speaker: So when you start doing this process, you really know, in this case for me, all 125 kids and what their levels are at to the point where I never had any concern at all.
Speaker: When I stepped out of the classroom, that was the beginning of the open AI era.
Speaker: cheating surveillance movement i never once thought to myself like oh is this open ai work or whatever because i you know already saw their stuff from like four weeks ago they've been working on it so it'd be really hard i mean it would almost be impressive if they were cheating yeah it reminds me of um i when i taught i taught uh an action research course this semester and at one point in the semester my students were documenting their learning um
Speaker: And at one point in the semester, I became concerned we were doing lit reviews.
Speaker: And it seemed to me that many of my students might have been using AI for the lit reviews.
Speaker: But it opened up this really powerful conversation with my department chair over, well, what's the purpose of the Action Research Project?
Speaker: It's to execute an intervention, a literacy intervention, and study how it performs.
Speaker: Well, isn't it really important then that they're interpreting the literature well?
Speaker: and that they're comprehending the research that they're doing as they're designing these interventions.
Speaker: Yeah.
Speaker: And if AI can help them do that well, and in a way that might be better than them doing it on their own, doesn't that lead to better action research outcomes?
Speaker: And I'm not absolutely certain that that's the case, but I'm pretty certain that that's how it bounced.
Speaker: And so my thinking changed as a result of them documenting their learning,
Speaker: Me noticing what I'm seeing in the lit reviews is very different than what I know of some of these writers and their work.
Speaker: But then ultimately, how they executed it in their classrooms and then produced findings, I honestly think it elevated the outcome in a way that I wasn't anticipating.
Speaker: And that was a great experience.
Speaker: Yeah, it's one of those things where it's very much a tool, not a replacement, similar to any other tool that's innovated on education.
Speaker: I don't think it's the exact same as, let's say, a calculator or Wikipedia.
Speaker: There's definitely different concerns, especially concerning data privacy and ethical use and things like that.
Speaker: But at the exact same time,
Speaker: I use chat GPT daily for some kind of mundane task, especially like starting an email or like finishing a thought I have because it's difficult for that.
Speaker: I have students, many who are pre-service special education teachers, they're 21 years old and heading into a classroom in the fall where they're going to have to rapidly design interventions for really complex kids.
Speaker: And they don't have a toolkit for that.
Speaker: I don't have a toolkit for it.
Speaker: any student that walks through my door.
Speaker: And so if we can talk with CHAP-GPT and get a sense of, I have a student who has these very specific learning disabilities, this is the context, what are 20 different ways in which I might make this a more inclusive experience for them?
Speaker: Could I come up with that on my own?
Speaker: Yes.
Speaker: Would it take me probably 24 hours to come up with 20 solutions?
Speaker: Yeah.
Speaker: And I can do it very rapidly.
Speaker: I think what's really important, though, now more than ever, is that we really are connected educators so that the 21-year-old is connected to people not just in their system, but outside of the system who can give feedback.
Speaker: And now we're going to take a quick break for a sponsored message.
Speaker: What if we started by listening?
Speaker: Human Restoration Project's empathy interview process involves students hosting peer conversations to uncover key themes that reveal the assets and challenges within schools and districts.
Speaker: Our work allows schools to hear from hundreds to thousands of students to listen, learn, and take action.
Speaker: Or in other words, we can literally hear from every single student on what they enjoy and what challenges they face at school.
Speaker: We do this through a three-step process.
Speaker: First, we train student facilitators to conduct empathy interviews or focus groups with their peers, gathering insights on what students love about school and what they wish to improve.
Speaker: all tailored to address the specific questions or interests of schools and districts.
Speaker: Second, we utilize cutting-edge conversation sense-making technology, qualitative data mapping software, and AI tools to code, theme, and analyze hundreds of student conversations.
Speaker: presenting key themes, student quotes, and areas of success and growth in a comprehensive school portal.
Speaker: And third, we provide tailored recommendations including research-backed strategies, actionable steps with varying levels of effort, and connections to additional partners to address key concerns.
Speaker: HRP supports districts with case studies, grant applications, and ongoing professional development to ensure successful implementation.
Speaker: This work is meant to elevate the incredible work that schools are doing while providing a never-before-possible opportunity to listen and learn from all young people in the learning community.
Speaker: Our goal is to make this process work with your space, and therefore we're highly flexible with time requirements, scheduling dates, district constraints, budgeting, and more.
Speaker: HRP has conducted hundreds of focus groups representing thousands of students in over 40 districts nationwide.
Speaker: We are proud to support strong school partnerships, equity-driven practices, and to support a strong data privacy policy to protect all learners.
Speaker: Our empathy interview process and ongoing workshops have retained a 94% great or excellent rating with educators, and we're honored that we are consistently invited back to districts to support their professional development.
Speaker: Learn more about this opportunity and get involved on our website, humanrestorationproject.org, and we look forward to hearing from you.
Speaker: What does it mean to capture, you literally say capture adjacent moments rather than, quote, the moment?
Speaker: What is the difference between those two things, adjacent versus the?
Speaker: So one of the things when I was very new to this work that would be overwhelming for me and sometimes pretty demoralizing is
Speaker: is I would enter into a learning experience hoping to learn something.
Speaker: And it is very difficult to be a teacher and a documentarian at the same time.
Speaker: It's difficult to document your learning while you're also teaching students.
Speaker: And one thing that people don't realize is that in Reggio Emilia, most teachers have pedagogistas or others in their company every day.
Speaker: They're doing the documentation work.
Speaker: And that's not the case in the West.
Speaker: And so it would be hard because I would go in with this purpose.
Speaker: And I was really excited to discover something or learning something.
Speaker: And I would be over here talking with a student or teaching something in a mini lesson.
Speaker: And it would happen, something that I wanted to document.
Speaker: And I couldn't grab it.
Speaker: And there's the sense that, oh, well, the moment's over.
Speaker: And there are a lot of people in the field who kind of speak to that by saying, well, you have to be prepared.
Speaker: You have to spend a lot of time before the lesson is taught, defining exactly what you're going to document, setting up the moment, making sure your technology is ready to go.
Speaker: And it's just, it doesn't work like that in my life.
Speaker: I need to be ready to document things on the fly.
Speaker: And oftentimes because I'm being present for my students, I don't grab it.
Speaker: But what can happen is when I'm done teaching my lesson,
Speaker: I can walk over there and take pictures of the work that's emerging or ask the student a follow-up question or capture a picture of something, you know, that a chart or I just, it might not even be particularly meaningful, but what I can do is try to document what still exists in the wake of the moment.
Speaker: And then later when I'm reflecting, I'm going to remember what that was that I wasn't able to capture from beginning to end.
Speaker: And the learning might be a little bit different, but it's still very valuable.
Speaker: also something that not only helps you and students, but it also helps, I would imagine, families and caregivers because they're able to see that work.
Speaker: Because I mean, there is something to be said about the learning process, as you mentioned earlier, that word journey.
Speaker: And a journey or wayfinding requires you to have connections with other folks, whether it be other students in the room, teachers, parents and caregivers, whoever that might be, even community members in this work.
Speaker: And it really, I think, helps demystify the learning process as not being something that's one and done, but as something that you're just doing.
Speaker: You're doing by being there.
Speaker: But what does that look like?
Speaker: big picture, how do you shift folks' perspective?
Speaker: How do you shift folks' perspective from this very standardized look of education towards something that is multimodal, recognizing just how ingrained that sense of
Speaker: Every nine weeks, we report a grade and that's what the grade is.
Speaker: And that's how we determine things like athletic eligibility and all these different things.
Speaker: These are questions that we run into a lot.
Speaker: We like everything that you're doing, but we're just going to adapt to the traditional system in order to make it work.
Speaker: So we'll use a portfolio, but you'll get a grade at the end of the nine weeks.
Speaker: In a perfect world, I'd like it to just be the portfolio, but I also recognize that that's not realistic.
Speaker: Is there work being done to help
Speaker: make that more holistic to shift?
Speaker: Well, I think that the sort of shift that you're talking about requires policy change in order to really make that work.
Speaker: And so I mentioned before initiatives like the New York state department of ed plan pilot, that is the policy change.
Speaker: It's one of them that, that might make a real difference there.
Speaker: At least that project might influence policy change.
Speaker: Right.
Speaker: And so I think to really get there, we need policy change.
Speaker: That said, if we flipped a switch tomorrow, policies change, and we have all of these teachers and leaders who have been dedicated to doing it the way we've always done it, that is not necessarily going to have the outcome that we want it to have, right?
Speaker: And so for me, what I find deeply rewarding, it's the best part of what I get to do,
Speaker: is I'm invited to come in and lead the shift to standards-based grading and reporting in a system.
Speaker: And usually that's happening more K-6 than it is at the middle school or high school.
Speaker: But when you shift to standards-based grading, people often think it's about the report card, and it's not.
Speaker: It's about pedagogy, and it's about the kind of assessment that's happening in the classroom.
Speaker: And really shifting practice to document learning, to focus more on formative, to move away from averages and more toward mode,
Speaker: to get real clear and start having conversations about what progress really looks like and a mastery scale.
Speaker: What's really lovely is when they begin sharing standards-based report cards with parents, and then they have all of this documentation to support the assessments that they're making, and parents and teachers start to value that.
Speaker: That's where I think the real shift happens.
Speaker: is when we're able to give a different level of information to students and students are able to gather a level of information that matters in those conversations and levels of concern and levels of joy, levels of concern come down, levels of celebration and joy start to emerge.
Speaker: And what really happens with documentation too is kids who might not necessarily be
Speaker: performing well in math, science, English, and social studies, when you start documenting their learning, you learn that they're great speakers or that they know how to draw and can do that really well, or that they are great story makers.
Speaker: And if they can build a story and they're doing that at home by, you know, they're building video games and they're coding and doing all the, what's so important about the documentation piece too is
Speaker: and moving in those directions is it illuminates what kids are good at in a way that's going to give them their career and their future.
Speaker: And so that's when parents and teachers and kids and leaders all start to understand the importance of moving in that direction.
Speaker: But I honestly, and, you know, people are going to disagree with me here.
Speaker: I honestly think that if we just flipped that switch tomorrow and,
Speaker: mandated that, you know, we all move in this direction.
Speaker: I don't think that that would go well because we got a lot of cultural shift to handle inside of the field.
Speaker: And something that I appreciate about the work that you're doing is that you're, it's not that we're blaming teachers.
Speaker: It's a stomach problem.
Speaker: And teachers need a lot of support to make those shifts.
Speaker: Just briefly, I remember when I was, I wrote this piece a while ago about kind of the history of the A to F grading scheme.
Speaker: And something that stuck out to me is that there's this letter from 1936, where it was a group of teachers of this random public school, and they actually stopped using letter grades for a series of years.
Speaker: And they shifted towards a narrative, like they would just write up what kids were doing, and they'd send it home.
Speaker: And it was mind blowing to everyone involved.
Speaker: They're like, the kids learn more.
Speaker: We loved it.
Speaker: It was amazing.
Speaker: And then they stopped, like just a few years later, because it wasn't sustainable.
Speaker: They found that there's no way that we could keep doing this.
Speaker: And that's the solution.
Speaker: These are the solutions we really need.
Speaker: And I truly believe this is where AI can help us.
Speaker: I don't know what the answers are, but I think that there's such potential there to make these more human forms of assessment manageable and more efficient without necessarily compromising the integrity of what we're doing.
Speaker: And I think that's exciting to think about.
Speaker: So then kind of moving into, I guess, like a final question.
Speaker: In what spaces do you see AI shifting that?
Speaker: Like, is this a certain tool, a certain prompt?
Speaker: How are you using AI in order to demystify that?
Speaker: Yeah.
Speaker: So I, the most powerful AI tool that I use is one that I've had in my pocket for many years now.
Speaker: It's called Otter AI.
Speaker: It's,
Speaker: And I'm sure there are others like it.
Speaker: They're not paying me to say anything.
Speaker: You can edit this out, of course.
Speaker: But I am using an AI augmented voice to text app that captures audio and narrative.
Speaker: And as I'm documenting my learning, I'm walking the room.
Speaker: I've just taught a lesson on, you know, how to create a claim for an argument.
Speaker: And students are working on that.
Speaker: They're building, they're drawing, they're talking, they're writing, and all sorts of different modalities going on.
Speaker: And I'm walking the room and I'm taking pictures and I'm capturing recordings.
Speaker: Otter is not only capturing voice to text, what I'm saying and what others are saying in the room.
Speaker: It's distinguishing speakers from one another and it's dropping the video and the pictures and audio into that larger transcript as I go.
Speaker: And then it codes all of that data for me.
Speaker: the minute that I hit end on the recording.
Speaker: So for me, that is the way that AI is enabling me and it's acting as an assistant inside of a really complex process where I'm capturing a ton of qualitative data and it would not be sustainable.
Speaker: If I was still teaching eighth grade and I had 130 students that I'm trying to do this with every day inside of 27 to 42 minute classes, which is what I had,
Speaker: Without a tool like that and the power of AI, I'm not sure I could move in that direction.
Speaker: And that's a totally realistic.
Speaker: So when people are saying, well, teachers should be doing it differently.
Speaker: No, this was absolutely impossible unless you had really small class sizes, really long classes, and some good assistants in the room helping you to document.
Speaker: And so I think the future is really bright here with these tools.
Speaker: Thank you again for listening to our podcast at Human Restoration Project.
Speaker: I hope this conversation leaves you inspired and ready to start making change.
Speaker: If you enjoyed listening, please consider leaving us a review on your favorite podcast player.
Speaker: Plus, find a whole host of free resources, writings, and other podcasts all for free on our website, humanrestorationproject.org.
Speaker: Thank you.
