Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Pedagogical Documentation w/ Angela Stockman
4 Plays9 months ago
Today we are joined by Angela Stockman. Angela is a veteran secondary English/Language Arts teacher, author, and professional learning facilitator. She has presented at state, national, and international levels and has led curriculum, assessment, and instructional design projects in over 100 school districts.
She has written books and resources on writing instruction, including The Writing Workshop Teacher's Guide to Multimodal Composition, Creating Inclusive Writing Environments in the K-12 Classroom, and the recently released The Writing Teacher’s Guide to Pedagogical Documentation: Rethinking How We Assess Learners and Learning, which we're talking about today.
Links:
Recommended
![Showcase: Spotlight on Bismarck Public Schools' Empower[Ed] image](https://media.zencastr.com/cdn-cgi/image/width=112,quality=85/image-files/5c7838c8f83db700141c53b1/7c62a589-872f-48c2-9bd6-a8fb6a3ae033.png)
Transcript
Introduction and Acknowledgments
00:00:10
Speaker
Hello and welcome to our latest episode of our podcast.
00:00:14
Speaker
My name is Chris McNutt and I'm part of the progressive education nonprofit Human Restoration Project.
00:00:19
Speaker
Before we get started, I want to let you know that this is brought to you by our supporters, three of whom are Peter Kratz, Janet Clark and Corinne Greenblatt.
00:00:27
Speaker
Thank you for your ongoing support.
00:00:28
Speaker
You can learn more about the Human Restoration Project on our website, humanrestorationproject.org or find us on social media and YouTube.
Introducing Angela Stockman and Her Work
00:00:36
Speaker
Today, we are joined by Angela Stockman.
00:00:38
Speaker
Angela is a veteran secondary English and language arts teacher, author, and professional learning facilitator.
00:00:44
Speaker
She has presented at state, national, and international levels and has led curriculum, assessment, and instructional design projects in over 100 different school districts.
00:00:52
Speaker
She has written books and resources on writing instruction, including the Writing Workshop Teacher's Guide to Multimodal Composition, Creating Inclusive Writing Environments in the K-12 Classroom,
00:01:02
Speaker
in the recently released The Writing Teacher's Guide to Pedagogical Documentation, Rethinking How We Assess Learners and Learning, which we're talking about today.
00:01:11
Speaker
So thank you, Angela, for joining us on the program.
00:01:13
Speaker
Oh, I feel so incredibly privileged to be here.
00:01:16
Speaker
Thanks, Chris.
00:01:18
Speaker
Yeah, let's just dive into it.
00:01:19
Speaker
Let's kick some things off.
00:01:20
Speaker
Let's talk about your most recent book, The Writing Teacher's Guide to Pedagogical Documentation.
Innovative Approaches to Learning Documentation
00:01:25
Speaker
And it has this really kick-ass hook in the book description.
00:01:29
Speaker
I was just checking out on Rutledge right before we jumped in.
00:01:32
Speaker
And it says, this book is a call to action for English and English language arts teachers who understand that data are not just numbers alone.
00:01:41
Speaker
learning is impossible to quantify and students are our very best teachers.
00:01:46
Speaker
So before we get into the specifics of the book, could you talk a little bit about what it means to document learning if it's impossible to quantify?
00:01:55
Speaker
Because I feel like it's very different than the norm.
00:01:57
Speaker
It is.
00:01:59
Speaker
And this has been quite a journey for me.
00:02:00
Speaker
I was excited to write this book because this is, I'm coming up on my 33rd year in the field.
00:02:07
Speaker
And from the moment I walked into my first classroom,
00:02:11
Speaker
I was very excited to teach writing, and it's been a huge passion of mine that sustained me throughout my career.
00:02:18
Speaker
I feel very fortunate to be the age that I am and still just as excited, if not more so, as I was when I was a brand new teacher that very first year.
00:02:30
Speaker
But yeah, historically, my experiences with assessment have had everything to do with
00:02:37
Speaker
measuring performance toward an outcome.
00:02:39
Speaker
There's been a heavy emphasis on summative assessments, more common forms that look like multiple choice or essay writing that we bring to a rubric in order to produce a grade.
00:02:52
Speaker
And that grade becomes the ultimate indicator of student success or failure.
00:02:58
Speaker
And the more that I've learned
00:03:00
Speaker
the more I've become interested in far more humane and human forms of assessment, which include pedagogical documentation.
00:03:10
Speaker
Pedagogical documentation is really about making learning visible.
AI in Education: Opportunities and Critiques
00:03:17
Speaker
And a lot of my work around that and my learning there has been inspired by Project Zero, Harvard's Project Zero, and Reggio-inspired educators who have understood this for a very long time, that assessment is a verb.
00:03:30
Speaker
It's something that we do.
00:03:32
Speaker
It's not something that we give.
00:03:35
Speaker
And when we make the learning process visible and we use all of these fabulous technology tools that we have at our disposal now to take pictures or capture audio or video or even just sit in conversation, quick conversations too, with learners, and they have the ability to make their thinking visible
00:03:58
Speaker
using representative materials.
00:04:00
Speaker
So for me, a lot of loose parts play, a lot of drawing, a lot of opportunity to talk or perform and not just reducing everything to written words and written numbers.
00:04:12
Speaker
We're actually able to learn so much more about students' strengths and their true needs.
00:04:19
Speaker
It makes us better diagnosticians too.
00:04:21
Speaker
So I find that when we rely on the numbers, um,
00:04:25
Speaker
it doesn't really tell me enough about what students truly are capable of.
00:04:30
Speaker
Um, and then I end up making the wrong assumptions often.
00:04:34
Speaker
And so documentation kind of allows us to make the full catastrophe of learning visible.
00:04:41
Speaker
Um, and we can scoop that information that we need, that we want to hold onto so that we can be far better, you know, assessors and more reflective.
00:04:51
Speaker
It makes that possible.
00:04:53
Speaker
And, um,
00:04:54
Speaker
For me, that is the most exciting and rewarding part of living during this time when we have such access to technology, including AI.
00:05:05
Speaker
AI gives us great potential there, too.
00:05:08
Speaker
More about in a second, because I'd love to ask a follow-up surrounding AI, because I wonder about some of the, I guess, the pushback that might happen.
00:05:16
Speaker
Let's just talk about that for a second.
00:05:17
Speaker
She's probably valid.
00:05:19
Speaker
I mean, what's interesting about AI is I was recently at a, well, I was at it with you.
00:05:24
Speaker
I was at NCTE.
00:05:25
Speaker
And one of the things that stood out to me at NCTE on the conference floor is the growth of AI grading tools, AI feedback tools.
00:05:34
Speaker
They're kind of taking the education world by storm.
00:05:37
Speaker
And what really stood out to me was that
00:05:40
Speaker
they're not just like nice to have, oh, the AI is going to help me write a comment on the student's work.
00:05:46
Speaker
It is very much like, here's the rubric, A, B, C, D, hit a bunch of check marks.
00:05:52
Speaker
Is that the right grade?
00:05:54
Speaker
Yes or no.
00:05:54
Speaker
And it's meant to speed up or make it more efficient.
00:05:57
Speaker
Yeah, we can make bad stuff more efficient.
00:06:00
Speaker
Yeah.
00:06:01
Speaker
So like, what does it look like to use AI in a more humane way?
00:06:06
Speaker
Because I feel like in educational, that's not the case, at least not right now.
00:06:09
Speaker
So I can give you an example in that three weeks ago, I brought K-12 teachers together over the course of five days.
00:06:16
Speaker
We were broken out into grade level cohorts.
00:06:18
Speaker
So K-2, 3, 5, 6, 12.
00:06:22
Speaker
And they brought to the table documentation that they gathered in their classroom in the form of pictures of students' work, audio recordings, transcripts.
00:06:34
Speaker
They brought all sorts of student work samples that were
00:06:38
Speaker
where kids were using loose parts to represent the parts of a story and then speaking over it in ways that had them using really high-level vocabulary words.
00:06:47
Speaker
And one of the biggest findings that came out of our work was that when students were using loose parts to represent their writing ideas, they were using high-level vocabulary words that they might not have initially had the print power for, but they were able to
00:07:03
Speaker
translate that like they were willing to hang in and include those words and metaphor and symbol in their writing because we took it one small step at a time.
00:07:13
Speaker
But the reason why we were able to draw these conclusions is I was using an app that captured 36 hours of audio and then pulled out the highlights, summarized our conversations so that I could put them back in front of everyone efficiently.
00:07:29
Speaker
It didn't take me because it usually has in the past two weeks.
00:07:33
Speaker
at home, some are all in that sort of data.
00:07:35
Speaker
I had it immediately.
00:07:38
Speaker
And I did code the data by hand.
00:07:40
Speaker
And what the AI tool did was absolutely in alignment with my own human coding work.
00:07:47
Speaker
And so to be able to instead of, and I truly believe that historically, we've reduced things to numbers that have diminished our understanding of what students know and are able to do
00:08:00
Speaker
because it's efficient and it's all we've been able to do.
00:08:03
Speaker
But while I'm not so much about plugging a rubric in into a tool and using it to generate comments on student work, and I'm not ever about putting student work into an AI tool without their consent to generate feedback, although it does a good job of that, I am all about tools that will help me capture on nearly 40 hours worth of audio.
00:08:28
Speaker
human conversations and it was sitting in my pocket.
00:08:30
Speaker
It was non-invasive.
00:08:32
Speaker
It allowed me to be more present.
00:08:34
Speaker
And those conversations could be a little bit meandering, which was necessary.
00:08:39
Speaker
And it produced conclusions rapidly that allowed us to really discover things that we wouldn't have otherwise.
00:08:48
Speaker
And that's the kind of AI I'm here.
00:08:51
Speaker
Yeah.
00:08:51
Speaker
I mean, I'm, I'm certainly a pro technology person too.
00:08:55
Speaker
Uh, I think a lot about the SAMR model, uh, where the last level of the SAMR model was, is redefinition.
00:09:01
Speaker
And something that's interesting about what you're talking about is that essentially that's something that you really couldn't do in the past.
00:09:07
Speaker
It would not be feasible for an average teacher to listen to 36 hours of audio recording, um, and actually make something of that.
00:09:15
Speaker
Um, however, do the growth in large language learning models, uh,
00:09:19
Speaker
You can.
00:09:19
Speaker
And that's really an impressive feat.
00:09:21
Speaker
And it's really cool.
00:09:23
Speaker
I can't help but self-plug.
00:09:24
Speaker
That's also how we do our PD.
00:09:25
Speaker
We essentially are recording hours upon hours of student voice work and then running it through an AI model to help get our themes.
00:09:33
Speaker
Obviously, there's a human component.
00:09:35
Speaker
We're not just like tossing it out there and just hoping that it works.
00:09:39
Speaker
But it would be you would need a team of hundreds of people to look to these results to do it any other way.
Documentation Methods and Strategies
00:09:46
Speaker
And you were there for, like, this is what I love too.
00:09:49
Speaker
And I'm sure it's your experience as well.
00:09:51
Speaker
I was present for those conversations.
00:09:53
Speaker
So when I'm looking at what the AI is generating in terms of conclusions, and I give it back to the team too, we're still in that whole experience.
00:10:05
Speaker
We're impacting it.
00:10:06
Speaker
We're evaluating it.
00:10:08
Speaker
We're deeply engaged with it all.
00:10:10
Speaker
And so we know whether or not it's doing what it needed to be doing.
00:10:14
Speaker
Yeah, yeah.
00:10:15
Speaker
Going back to just like traditional, I guess, pedagogical documentation, steering it back to that part of the conversation.
00:10:22
Speaker
Something that you were saying reminded me of when I was teaching.
00:10:25
Speaker
It was early in my teaching career, like year three, year four.
00:10:28
Speaker
And I remember I used to give a lot of multiple choice tests because it was very efficient.
00:10:33
Speaker
And I was a history teacher at the time.
00:10:35
Speaker
It was like 10 question multiple choice tests every Friday.
00:10:39
Speaker
And I remember I used to have these students who I thought were doing very well in class, seemed to always participate, always knew the right answers, etc.
00:10:47
Speaker
And
00:10:49
Speaker
they would get like a five out of 10 on these tests.
00:10:51
Speaker
And it didn't occur to me sadly until year three or year four to ask them, okay, why are we not getting them?
00:10:58
Speaker
So I remember I pulled these students aside and they were questioned surrounding, I think the civil rights movement.
00:11:04
Speaker
So I'd ask them, you know, like, oh, well, like you put here, the Martin Luther King Jr. thought this, or that Malcolm X thought this, et cetera.
00:11:09
Speaker
And then they would be able to explain the entirely right answer to me, even though that's not what they marked on the multiple choice test.
00:11:16
Speaker
And that led to this conundrum where, first off, I had a little bit of an existential crisis because I thought about all the kids I probably gave Ds or Fs to, thinking that I was doing the right thing, but they probably knew the right answer.
00:11:26
Speaker
But also, it led to some introspective thinking where we could figure out different ways of assessment.
00:11:33
Speaker
So what are the other ways that we could capture those moments that you write about a lot in the work?
00:11:37
Speaker
we landed on portfolios, but I'm interested to learn more about the examples that you share.
00:11:43
Speaker
So there's everything.
00:11:44
Speaker
There's like the photo voice method, there's a documentation notebook.
00:11:48
Speaker
What does it actually look like to capture student learning in this way?
00:11:51
Speaker
So I think from my position inside of this work, I'm usually a facilitator who's brought in to work with people who are coming from very different places.
00:12:01
Speaker
I have very inexperienced teachers.
00:12:04
Speaker
I have some veteran teachers.
00:12:05
Speaker
I work in very different
00:12:07
Speaker
parts of the world.
00:12:09
Speaker
And so the first thing that I think of is comfort level and how brittle the teacher is that I'm working with.
00:12:19
Speaker
And brittleness is usually the effect of being completely overwhelmed in their work.
00:12:25
Speaker
And so when I'm working with a teacher who's basically walking in the door going, girlfriend, I have no time for this.
00:12:33
Speaker
I need to be very realistic about
00:12:35
Speaker
What sort of documentation approach is really going to help this teacher gain traction and really fall in love with the process because it makes that teacher feel far more effective at what they're doing.
00:12:50
Speaker
And so early on, it might be something as simple as documenting a moment.
00:12:56
Speaker
And my friend Aaron Shorn and Will Rupin at Unruler talk about it in that particular way, like,
00:13:03
Speaker
If there's something very specific you want to learn in a lesson, maybe you're only going to document one moment in that lesson.
00:13:11
Speaker
Maybe you're going to document your direct instruction.
00:13:14
Speaker
Maybe you're going to document conversation that you have with a student, one student even.
00:13:19
Speaker
So it can be as simple as documenting what a student says, what a student does, or what a learner produces in a learning moment, or yourself, what you said, what you did.
00:13:32
Speaker
what you produced in a moment by taking a picture.
00:13:35
Speaker
And I do this all week long where I will take a ton of pictures and I'll wait until the weekend or when I have a free moment, I literally have next week where I'm going to be sitting with probably over a thousand photos I took over the last month because I haven't had time.
00:13:50
Speaker
And then start to interpret them.
00:13:53
Speaker
Documentation can also look like a journey where maybe you're going to document how you're teaching and
00:14:00
Speaker
inside of a writing mini lesson over the course of a week or a month or even an entire school year.
00:14:07
Speaker
When I work with teachers around improving performance, they might have identified in one district that I'm working in right now, it's become very evident.
00:14:16
Speaker
We have a lot of evidence that's suggesting that regardless of the grade level, students are struggling to both identify and express main ideas in their writing.
00:14:28
Speaker
That's pretty significant because the ability to do that impacts a lot of things as readers and writers and learning human beings.
00:14:37
Speaker
So we've made that a focus for the entire year and we're not documenting everything.
00:14:43
Speaker
We're documenting how we're teaching that concept.
00:14:46
Speaker
We're documenting how students are thinking about it and we're documenting what they produce and we're bringing that data back to the table.
00:14:54
Speaker
And so I think it's really important to have a focus
00:14:58
Speaker
to understand that it can be something micro, like just a moment, and you're going to take pictures or capture one video.
00:15:05
Speaker
But then usually what starts to happen is you become comfortable.
00:15:09
Speaker
It becomes sort of innate in your practice and not overwhelming.
00:15:14
Speaker
And it's deeply rewarding to do that.
00:15:15
Speaker
Portfolios, you know, are beautiful.
00:15:19
Speaker
I'm excited because I was just invited to serve as an assessment facilitator on the New York State Plan Pilot team.
00:15:27
Speaker
New York State is piloting with dozens of school districts the option to provide students the opportunity to pursue portfolio performance-based, project-based pathways instead of in lieu of the regents exams.
00:15:44
Speaker
So I'm super excited to be a part of that pilot.
00:15:48
Speaker
Portfolios can be big, right?
00:15:51
Speaker
And that can also be something that students are polishing up and having to do a lot of revision work around.
00:15:59
Speaker
Documentation is more about the process.
00:16:02
Speaker
And you can have a process-based portfolio as well.
00:16:05
Speaker
And I honestly think that that's a lot of what emerges at the end of a documentation project.
00:16:11
Speaker
I had my students in my assessment methods course at Damon University document their learning for the entire semester.
00:16:18
Speaker
And at the end, what they basically have is a portfolio.
00:16:21
Speaker
But then I think of my daughter, Laura, who's a designer and her partner, Zach, he's an industrial designer and they have these gorgeous portfolios that they have posted online so that potential clients can uprope their work.
00:16:37
Speaker
So portfolio has some different sorts of translations, but yeah, it can be something very small.
00:16:45
Speaker
And for me, when I'm documenting my learning,
00:16:49
Speaker
I'm often moving at the speed of light and I want to be really present for my students.
00:16:53
Speaker
So I'm just capturing things and I'm not stopping in the moment, you know, necessarily to reflect deeply, but I'm taking a picture, I'm grabbing a video, I'm grabbing student work, knowing that when I sit down to look at and interpret it, it's going to be deeply meaningful.
00:17:10
Speaker
Yeah.
00:17:10
Speaker
And it sounds like too, in a lot of the situations that you name in the book, students are the ones that are selecting the
00:17:18
Speaker
which forms of documentation best represent their learning.
00:17:21
Speaker
So in many cases, it's the student who's taking the pictures or reflecting on the pictures that are provided.
00:17:26
Speaker
And is there a meta component to that?
00:17:29
Speaker
So I wonder if one of the things, for example, that English teachers are looking to do is having students identify main ideas in their writing.
00:17:37
Speaker
that also means that they would be identifying main ideas and they're learning through the documentation process.
00:17:42
Speaker
Yes.
00:17:42
Speaker
So how do we get like more meta with that?
00:17:45
Speaker
What are the skills that we're learning by doing pedagogy in this way?
00:17:48
Speaker
Well, I think that what's really beautiful, and it's something that's hard to translate for teachers that are not involved in these conversations often.
00:17:59
Speaker
The beautiful thing about pedagogical documentation is it relies on multimodal expression.
00:18:05
Speaker
So the thinking is that
00:18:07
Speaker
We're not just going to rely on alphabetic text and print numbers and letter grades to tell us what students know and are able to do.
00:18:15
Speaker
We're going to invite them to perform or draw or build something.
00:18:20
Speaker
And that enables invites them to use different modes of expression.
00:18:25
Speaker
It requires us to use different modes of documentation.
00:18:30
Speaker
And it also relies on conceptual transfer.
00:18:35
Speaker
And so when we start talking about going meta with all of this and trying to explain to students what, here's what a main idea might be.
00:18:45
Speaker
Do you notice it in a story?
00:18:47
Speaker
Do you notice it in this photo?
00:18:49
Speaker
Do you notice it in this video?
00:18:51
Speaker
Do you notice it in this animated short that you just watched?
00:18:54
Speaker
Now let's look at your work.
00:18:56
Speaker
What's the main idea of your work?
00:18:58
Speaker
And so what's gorgeous is that
00:19:01
Speaker
More than any other assessment method I've ever used, pedagogical documentation really facilitates another level of transfer, an added layer, and that can be integrated into instructional design in ways that I feel is very sort of organic, where
Transformative Impact of Documentation
00:19:22
Speaker
other forms of assessment, the stuff that we've traditionally used, like multiple choice and essay, you're kind of bringing the learning to a halt.
00:19:30
Speaker
in order to test whether or not it happened.
00:19:33
Speaker
And then you're getting this lousy data typically that requires you to do more inquiry anyway.
00:19:39
Speaker
For me, documentation is a beautifully integrative approach to assessment because it too is multimodal in nature and it really plays well with teaching for transfer.
00:19:53
Speaker
it's one of those things to me that just feels more authentic and real in the quote unquote real world, which the school is the real world, but school is one of the very few places where you're ever going to consistently take quizzes and tests or a document, the things that, you know, you might take one of those at the end of like college, like you might have your bar exam or something, but,
00:20:12
Speaker
For the most part, it's not common.
00:20:15
Speaker
Or if it does happen, it's used for the most basic things that no one cares about.
00:20:19
Speaker
Like I think about safe schools for teachers where it's like, what's the right chemical thing to use on this type of fire?
00:20:26
Speaker
I don't know.
00:20:26
Speaker
There's all these strange multiple choice tests that we have to take.
00:20:30
Speaker
But beyond that, you are doing multimodal expression.
00:20:33
Speaker
You are having folks come in and observe the things that you're doing to tell a story about how you do your job.
00:20:40
Speaker
Or people might, I mean, actually take pictures and videos of your work and use that in order to do improvement plans, et cetera.
00:20:46
Speaker
It just seems like it would prepare students for not only, it's going to prepare students not only to learn more, but just to have more robust learning experiences down the road because they're used to it.
00:20:57
Speaker
They're going to understand.
00:20:58
Speaker
Yeah.
00:20:59
Speaker
What I love.
00:21:00
Speaker
So I use a lot of different tools for documentation, depending on the context that I'm in.
00:21:04
Speaker
But one of the tools that I use is Unruler, Aaron Shorn.
00:21:08
Speaker
And I know that, that, you know, Aaron and probably.
00:21:11
Speaker
And so Aaron, one of the beautiful things that I love about Unruler is it looks a lot like Instagram.
00:21:25
Speaker
And when I'm working with my students in Unruler, they start to, and documentation does this as well, they start to realize that learning moments are just as worth remembering as maybe that trip I took with my friends last week.
00:21:43
Speaker
And it sort of starts to unschool the learning experience and turn it into something that's worth building memories around.
00:21:55
Speaker
And that can be incredibly empowering and rewarding for students, particularly those who might have sort of been beaten up with grades in the past, right?
00:22:08
Speaker
And they start to realize Maggie Burns, who is my former student, she's an intern now for Aaron and Will, focused deeply on documenting her learning, illuminated a really bad relationship that she had with perfectionism.
00:22:23
Speaker
And she kept documenting her learning after my class was over because she wanted to, because it was helping her learn things about herself.
00:22:32
Speaker
I think what all of us who are connected online and who have been connected educators through social media have been doing that has brought us into relationship with each other is we've been documenting our learning and sharing it on Twitter and LinkedIn and Instagram.
00:22:48
Speaker
So that is a really beautiful thing.
00:22:52
Speaker
And I really think it starts to shift how we value learning and where we value learning outside of the walls of school.
00:23:04
Speaker
Testing, all of those tests, really, we give them because teachers want to have certainty and leaders want to have certainty that some basic things have been learned.
00:23:15
Speaker
And that's okay.
00:23:17
Speaker
There's a time and place for multiple choice.
00:23:19
Speaker
I'm not going to totally diss it.
00:23:21
Speaker
There are times when I will use selected response because I just want to know, you know, do they know the basics?
00:23:27
Speaker
But yeah, I think documentation helps me do so much more in students too.
00:23:33
Speaker
Yeah, it gets to the heart of something that I've always been obsessed with, which is I'm sure that you find when you change the assessment systems in a class, it has the additional effect of changing the entire pedagogical framework of what a teacher is doing.
00:23:50
Speaker
Because at the end of the day, if I'm a more...
00:23:53
Speaker
traditional legacy, whatever word you want to use to describe a teacher who does more of like the PowerPoint multiple choice test question format.
00:24:01
Speaker
That's not a very exciting thing to document because all I'm really documenting is some notes.
00:24:07
Speaker
And then the next day I'm documenting more notes.
00:24:09
Speaker
Yeah.
00:24:09
Speaker
And you can document and I have how items perform, can unwrap test items and get at content, context, cognitive load, those sorts of things.
00:24:19
Speaker
And I do that sometimes too, but
00:24:23
Speaker
That's really about analyzing how items perform.
00:24:26
Speaker
That's not really documenting significant shifts in our thinking, our learning, our work, or the culture.
00:24:34
Speaker
And that's where the good stuff is.
00:24:36
Speaker
Yeah.
00:24:36
Speaker
So then do you find as folks work through the workshop that they start thinking to themselves, well, I know kids are going to be documenting all these cool things that they're doing.
00:24:45
Speaker
That means I need to make more cool experiences like project-based learning in order to make that happen.
00:24:50
Speaker
Like, does it lead to the, yeah.
00:24:52
Speaker
Yeah, I think sometimes what's interesting is I think sometimes there's a perspective around beginning work with pedagogical documentation.
00:25:02
Speaker
And I'm often thinking about this as a facilitator.
00:25:06
Speaker
Sometimes there's a perspective that, you know, I walk in a door and we start documenting our learning and all of a sudden everybody, you know, there's fireworks and confetti and everyone's just living in this new peaceable kingdom that's so
Challenges of Shifting Assessment Methods
00:25:19
Speaker
ultra creative.
00:25:19
Speaker
That is not at all what it's like.
00:25:21
Speaker
I often find that students and teachers alike, there's some scaffolding that goes on into that work because it's very uncomfortable.
00:25:33
Speaker
And yeah, one of the conversations I have to have often is, what does it feel like when we're learning?
00:25:41
Speaker
And I know that we have these, and I said this last week in sessions that I was in, we have these very cliched conversations about learning is frustrating and
00:25:49
Speaker
And failure is a part of it.
00:25:50
Speaker
And you want to fail fast.
00:25:52
Speaker
And we've been having that conversation for like 20 years now in the field.
00:25:56
Speaker
I'm really talking about a human being, an individual, understanding the embodied experience of learning that is very unique to them.
00:26:07
Speaker
And I think that we get disconnected from that very rapidly.
00:26:11
Speaker
And unless you understand that feeling for yourself, you don't know when to document your learning.
00:26:19
Speaker
unless you're being really directed to do it at a certain moment.
00:26:23
Speaker
And so in the beginning, it does feel like teachers are selecting and students are selecting very specific moments to pause, reflect, and document.
00:26:36
Speaker
But where you want to get eventually is to a place where it is intuitive because I am self-aware and I know when I am learning and what that feels like for me.
00:26:47
Speaker
And that is changing how I learn.
00:26:49
Speaker
And when that starts happening, then you start to see a shift in teachers' relationships with students and students' ownership and influence over the curriculum and also how things are being taught.
00:27:02
Speaker
That takes time.
00:27:04
Speaker
It doesn't happen in the first two weeks, right?
00:27:07
Speaker
It is definitely a process.
00:27:09
Speaker
I have made that mistake.
00:27:11
Speaker
I remember when I first shifted over to portfolios, I got really into the idea of unschooling the classroom.
00:27:18
Speaker
And the very first thing I did was like, well, I'll just put, you know, 20 standards up and kids will just tell me whenever they learn about those standards.
00:27:26
Speaker
And that's it.
00:27:27
Speaker
It'll be great.
00:27:28
Speaker
It'll work out fantastic.
00:27:29
Speaker
And I learned right away that does not work because first off kids freaked out because like
00:27:34
Speaker
How am I supposed to know which one of these things goes in which standard?
00:27:37
Speaker
It's like you're ripping the rug out from under them.
00:27:41
Speaker
Someone has always that's always been the teacher's job.
00:27:44
Speaker
They haven't paid attention to it.
00:27:46
Speaker
They haven't been developing that facility in any way.
00:27:50
Speaker
Right.
00:27:50
Speaker
I also didn't have any resources prepared to help them.
00:27:54
Speaker
Like I didn't anticipate how much work it would take to have that scaffolded journey.
00:28:00
Speaker
Yes.
00:28:01
Speaker
You know, by the time I left the classroom, that was an ongoing process that happened at least once a week.
00:28:07
Speaker
We did portfolios.
00:28:09
Speaker
But we did not start there.
00:28:11
Speaker
It very much started like where you're saying, where it's like, hey, on Friday, we're going to talk about what we learned this week.
00:28:16
Speaker
And here's like one standard.
00:28:18
Speaker
Which one do you think that goes with?
00:28:19
Speaker
And it was very much like an elementary look at how do we choose the things that we're learning.
00:28:24
Speaker
But it does get to that point where kids, like one of my most proud moments concerning portfolios is when kids would do something in another class and then come up to me and say, can I use this thing I did in my fine class for your standard?
00:28:37
Speaker
I was like,
00:28:38
Speaker
yeah, that's amazing.
00:28:40
Speaker
And then a bunch of kids would replicate it.
00:28:41
Speaker
And then you just have like a lot of people.
00:28:42
Speaker
Yeah.
00:28:43
Speaker
And then it starts to roll.
00:28:44
Speaker
That's where you really start to get traction that moves you toward down a new bend in the whole path.
00:28:50
Speaker
That's really exciting.
00:28:53
Speaker
Conversations that I sometimes have with teachers and administrators too, where they'll, there will start to be a lot of frustration expressed around standardized testing.
00:29:04
Speaker
You know, we spend all of this time preparing for the tests and executing the tests and grading the tests.
00:29:10
Speaker
And we just want to do project-based learning and we want to use portfolios because the time alone.
00:29:16
Speaker
And I'm like, okay, let's take a step back here because if time is really the factor we're playing with here, you may not be satisfied because actually this approach takes just as much, if not more time,
00:29:33
Speaker
to execute well and to make meaningful.
00:29:35
Speaker
And I think it, but what I find is it just keeps us all teachers too engaged in the work and rewarded by the work.
00:29:45
Speaker
And if you can be intellectually curious and not take it personally, when things don't go well, you know, when you put the 20 standards up and the kids freak out, you didn't quit and just say, well, this isn't going to work.
00:29:59
Speaker
No, you kind of leaned into it like, all right, how am I going to, it becomes a fun problem to solve in some ways too.
00:30:06
Speaker
And that's really the spirit that you have to come at all of this with in order for it to be rewarding.
00:30:11
Speaker
Yeah, it helps you connect with people too.
00:30:14
Speaker
Like that to me was the big standout moment was that I think I was always pretty good at having like positive relationships with kids, but not necessarily connecting with them at the learner level.
00:30:23
Speaker
So when you start doing this process, you really know, in this case for me, all 125 kids and what their levels are at to the point where I never had any concern at all.
00:30:35
Speaker
When I stepped out of the classroom, that was the beginning of the open AI era.
00:30:40
Speaker
cheating surveillance movement i never once thought to myself like oh is this open ai work or whatever because i you know already saw their stuff from like four weeks ago they've been working on it so it'd be really hard i mean it would almost be impressive if they were cheating yeah it reminds me of um i when i taught i taught uh an action research
AI's Role in Future Assessments
00:31:01
Speaker
course this semester and at one point in the semester my students were documenting their learning um
00:31:07
Speaker
And at one point in the semester, I became concerned we were doing lit reviews.
00:31:12
Speaker
And it seemed to me that many of my students might have been using AI for the lit reviews.
00:31:18
Speaker
But it opened up this really powerful conversation with my department chair over, well, what's the purpose of the Action Research Project?
00:31:25
Speaker
It's to execute an intervention, a literacy intervention, and study how it performs.
00:31:29
Speaker
Well, isn't it really important then that they're interpreting the literature well?
00:31:35
Speaker
and that they're comprehending the research that they're doing as they're designing these interventions.
00:31:40
Speaker
Yeah.
00:31:41
Speaker
And if AI can help them do that well, and in a way that might be better than them doing it on their own, doesn't that lead to better action research outcomes?
00:31:51
Speaker
And I'm not absolutely certain that that's the case, but I'm pretty certain that that's how it bounced.
00:31:59
Speaker
And so my thinking changed as a result of them documenting their learning,
00:32:05
Speaker
Me noticing what I'm seeing in the lit reviews is very different than what I know of some of these writers and their work.
00:32:11
Speaker
But then ultimately, how they executed it in their classrooms and then produced findings, I honestly think it elevated the outcome in a way that I wasn't anticipating.
00:32:23
Speaker
And that was a great experience.
00:32:24
Speaker
Yeah, it's one of those things where it's very much a tool, not a replacement, similar to any other tool that's innovated on education.
00:32:34
Speaker
I don't think it's the exact same as, let's say, a calculator or Wikipedia.
00:32:38
Speaker
There's definitely different concerns, especially concerning data privacy and ethical use and things like that.
00:32:44
Speaker
But at the exact same time,
00:32:46
Speaker
I use chat GPT daily for some kind of mundane task, especially like starting an email or like finishing a thought I have because it's difficult for that.
00:32:58
Speaker
I have students, many who are pre-service special education teachers, they're 21 years old and heading into a classroom in the fall where they're going to have to rapidly design interventions for really complex kids.
00:33:10
Speaker
And they don't have a toolkit for that.
00:33:12
Speaker
I don't have a toolkit for it.
00:33:15
Speaker
any student that walks through my door.
00:33:17
Speaker
And so if we can talk with CHAP-GPT and get a sense of, I have a student who has these very specific
Empathy Interviews and Multimodal Approaches
00:33:25
Speaker
learning disabilities, this is the context, what are 20 different ways in which I might make this a more inclusive experience for them?
00:33:32
Speaker
Could I come up with that on my own?
00:33:34
Speaker
Yes.
00:33:35
Speaker
Would it take me probably 24 hours to come up with 20 solutions?
00:33:40
Speaker
Yeah.
00:33:41
Speaker
And I can do it very rapidly.
00:33:44
Speaker
I think what's really important, though, now more than ever, is that we really are connected educators so that the 21-year-old is connected to people not just in their system, but outside of the system who can give feedback.
00:34:00
Speaker
And now we're going to take a quick break for a sponsored message.
00:34:05
Speaker
What if we started by listening?
00:34:07
Speaker
Human Restoration Project's empathy interview process involves students hosting peer conversations to uncover key themes that reveal the assets and challenges within schools and districts.
00:34:19
Speaker
Our work allows schools to hear from hundreds to thousands of students to listen, learn, and take action.
00:34:26
Speaker
Or in other words, we can literally hear from every single student on what they enjoy and what challenges they face at school.
00:34:33
Speaker
We do this through a three-step process.
00:34:35
Speaker
First, we train student facilitators to conduct empathy interviews or focus groups with their peers, gathering insights on what students love about school and what they wish to improve.
00:34:45
Speaker
all tailored to address the specific questions or interests of schools and districts.
00:34:49
Speaker
Second, we utilize cutting-edge conversation sense-making technology, qualitative data mapping software, and AI tools to code, theme, and analyze hundreds of student conversations.
00:35:00
Speaker
presenting key themes, student quotes, and areas of success and growth in a comprehensive school portal.
00:35:07
Speaker
And third, we provide tailored recommendations including research-backed strategies, actionable steps with varying levels of effort, and connections to additional partners to address key concerns.
00:35:17
Speaker
HRP supports districts with case studies, grant applications, and ongoing professional development to ensure successful implementation.
00:35:25
Speaker
This work is meant to elevate the incredible work that schools are doing while providing a never-before-possible opportunity to listen and learn from all young people in the learning community.
00:35:35
Speaker
Our goal is to make this process work with your space, and therefore we're highly flexible with time requirements, scheduling dates, district constraints, budgeting, and more.
00:35:45
Speaker
HRP has conducted hundreds of focus groups representing thousands of students in over 40 districts nationwide.
00:35:51
Speaker
We are proud to support strong school partnerships, equity-driven practices, and to support a strong data privacy policy to protect all learners.
00:35:59
Speaker
Our empathy interview process and ongoing workshops have retained a 94% great or excellent rating with educators, and we're honored that we are consistently invited back to districts to support their professional development.
00:36:10
Speaker
Learn more about this opportunity and get involved on our website, humanrestorationproject.org, and we look forward to hearing from you.
00:36:18
Speaker
What does it mean to capture, you literally say capture adjacent moments rather than, quote, the moment?
00:36:27
Speaker
What is the difference between those two things, adjacent versus the?
00:36:31
Speaker
So one of the things when I was very new to this work that would be overwhelming for me and sometimes pretty demoralizing is
00:36:39
Speaker
is I would enter into a learning experience hoping to learn something.
00:36:44
Speaker
And it is very difficult to be a teacher and a documentarian at the same time.
00:36:51
Speaker
It's difficult to document your learning while you're also teaching students.
00:36:56
Speaker
And one thing that people don't realize is that in Reggio Emilia, most teachers have pedagogistas or others in their company every day.
00:37:07
Speaker
They're doing the documentation work.
00:37:09
Speaker
And that's not the case in the West.
00:37:11
Speaker
And so it would be hard because I would go in with this purpose.
00:37:14
Speaker
And I was really excited to discover something or learning something.
00:37:17
Speaker
And I would be over here talking with a student or teaching something in a mini lesson.
00:37:24
Speaker
And it would happen, something that I wanted to document.
00:37:27
Speaker
And I couldn't grab it.
00:37:28
Speaker
And there's the sense that, oh, well, the moment's over.
00:37:32
Speaker
And there are a lot of people in the field who kind of speak to that by saying, well, you have to be prepared.
00:37:37
Speaker
You have to spend a lot of time before the lesson is taught, defining exactly what you're going to document, setting up the moment, making sure your technology is ready to go.
00:37:46
Speaker
And it's just, it doesn't work like that in my life.
00:37:50
Speaker
I need to be ready to document things on the fly.
00:37:53
Speaker
And oftentimes because I'm being present for my students, I don't grab it.
00:37:57
Speaker
But what can happen is when I'm done teaching my lesson,
00:38:02
Speaker
I can walk over there and take pictures of the work that's emerging or ask the student a follow-up question or capture a picture of something, you know, that a chart or I just, it might not even be particularly meaningful, but what I can do is try to document what still exists in the wake of the moment.
00:38:23
Speaker
And then later when I'm reflecting, I'm going to remember what that was that I wasn't able to capture from beginning to end.
00:38:31
Speaker
And the learning might be a little bit different, but it's still very valuable.
00:38:35
Speaker
also something that not only helps you and students, but it also helps, I would imagine, families and caregivers because they're able to see that work.
00:38:43
Speaker
Because I mean, there is something to be said about the learning process, as you mentioned earlier, that word journey.
00:38:50
Speaker
And a journey or wayfinding requires you to have connections with other folks, whether it be other students in the room, teachers, parents and caregivers, whoever that might be, even community members in this work.
00:39:04
Speaker
And it really, I think, helps demystify the learning process as not being something that's one and done, but as something that you're just doing.
00:39:12
Speaker
You're doing by being there.
00:39:15
Speaker
But what does that look like?
00:39:18
Speaker
big picture, how do you shift folks' perspective?
00:39:22
Speaker
How do you shift folks' perspective from this very standardized look of education towards something that is multimodal, recognizing just how ingrained that sense of
00:39:37
Speaker
Every nine weeks, we report a grade and that's what the grade is.
00:39:40
Speaker
And that's how we determine things like athletic eligibility and all these different things.
00:39:44
Speaker
These are questions that we run into a lot.
00:39:47
Speaker
We like everything that you're doing, but we're just going to adapt to the traditional system in order to make it work.
00:39:51
Speaker
So we'll use a portfolio, but you'll get a grade at the end of the nine weeks.
00:39:54
Speaker
In a perfect world, I'd like it to just be the portfolio, but I also recognize that that's not realistic.
00:40:00
Speaker
Is there work being done to help
00:40:02
Speaker
make that more holistic to shift?
00:40:04
Speaker
Well, I think that the sort of shift that you're talking about requires policy change in order to really make that work.
00:40:14
Speaker
And so I mentioned before initiatives like the New York state department of ed plan pilot, that is the policy change.
00:40:22
Speaker
It's one of them that, that might make a real difference there.
00:40:27
Speaker
At least that project might influence policy change.
00:40:30
Speaker
Right.
00:40:31
Speaker
And so I think to really get there, we need policy change.
00:40:34
Speaker
That said, if we flipped a switch tomorrow, policies change, and we have all of these teachers and leaders who have been dedicated to doing it the way we've always done it, that is not necessarily going to have the outcome that we want it to have, right?
00:40:53
Speaker
And so for me, what I find deeply rewarding, it's the best part of what I get to do,
00:40:59
Speaker
is I'm invited to come in and lead the shift to standards-based grading and reporting in a system.
00:41:05
Speaker
And usually that's happening more K-6 than it is at the middle school or high school.
00:41:09
Speaker
But when you shift to standards-based grading, people often think it's about the report card, and it's not.
00:41:14
Speaker
It's about pedagogy, and it's about the kind of assessment that's happening in the classroom.
00:41:20
Speaker
And really shifting practice to document learning, to focus more on formative, to move away from averages and more toward mode,
00:41:29
Speaker
to get real clear and start having conversations about what progress really looks like and a mastery scale.
00:41:36
Speaker
What's really lovely is when they begin sharing standards-based report cards with parents, and then they have all of this documentation to support the assessments that they're making, and parents and teachers start to value that.
00:41:56
Speaker
That's where I think the real shift happens.
00:41:59
Speaker
is when we're able to give a different level of information to students and students are able to gather a level of information that matters in those conversations and levels of concern and levels of joy, levels of concern come down, levels of celebration and joy start to emerge.
00:42:21
Speaker
And what really happens with documentation too is kids who might not necessarily be
00:42:29
Speaker
performing well in math, science, English, and social studies, when you start documenting their learning, you learn that they're great speakers or that they know how to draw and can do that really well, or that they are great story makers.
00:42:43
Speaker
And if they can build a story and they're doing that at home by, you know, they're building video games and they're coding and doing all the, what's so important about the documentation piece too is
00:42:56
Speaker
and moving in those directions is it illuminates what kids are good at in a way that's going to give them their career and their future.
00:43:05
Speaker
And so that's when parents and teachers and kids and leaders all start to understand the importance of moving in that direction.
Policy Changes and AI Efficiency
00:43:15
Speaker
But I honestly, and, you know, people are going to disagree with me here.
00:43:19
Speaker
I honestly think that if we just flipped that switch tomorrow and,
00:43:23
Speaker
mandated that, you know, we all move in this direction.
00:43:26
Speaker
I don't think that that would go well because we got a lot of cultural shift to handle inside of the field.
00:43:31
Speaker
And something that I appreciate about the work that you're doing is that you're, it's not that we're blaming teachers.
00:43:36
Speaker
It's a stomach problem.
00:43:37
Speaker
And teachers need a lot of support to make those shifts.
00:43:42
Speaker
Just briefly, I remember when I was, I wrote this piece a while ago about kind of the history of the A to F grading scheme.
00:43:50
Speaker
And something that stuck out to me is that there's this letter from 1936, where it was a group of teachers of this random public school, and they actually stopped using letter grades for a series of years.
00:44:03
Speaker
And they shifted towards a narrative, like they would just write up what kids were doing, and they'd send it home.
00:44:08
Speaker
And it was mind blowing to everyone involved.
00:44:10
Speaker
They're like, the kids learn more.
00:44:11
Speaker
We loved it.
00:44:11
Speaker
It was amazing.
00:44:12
Speaker
And then they stopped, like just a few years later, because it wasn't sustainable.
00:44:17
Speaker
They found that there's no way that we could keep doing this.
00:44:19
Speaker
And that's the solution.
00:44:20
Speaker
These are the solutions we really need.
00:44:22
Speaker
And I truly believe this is where AI can help us.
00:44:28
Speaker
I don't know what the answers are, but I think that there's such potential there to make these more human forms of assessment manageable and more efficient without necessarily compromising the integrity of what we're doing.
00:44:45
Speaker
And I think that's exciting to think about.
00:44:48
Speaker
So then kind of moving into, I guess, like a final question.
00:44:52
Speaker
In what spaces do you see AI shifting that?
00:44:55
Speaker
Like, is this a certain tool, a certain prompt?
00:44:59
Speaker
How are you using AI in order to demystify that?
00:45:02
Speaker
Yeah.
00:45:03
Speaker
So I, the most powerful AI tool that I use is one that I've had in my pocket for many years now.
00:45:10
Speaker
It's called Otter AI.
00:45:11
Speaker
It's,
00:45:12
Speaker
And I'm sure there are others like it.
00:45:14
Speaker
They're not paying me to say anything.
00:45:15
Speaker
You can edit this out, of course.
00:45:17
Speaker
But I am using an AI augmented voice to text app that captures audio and narrative.
00:45:28
Speaker
And as I'm documenting my learning, I'm walking the room.
00:45:32
Speaker
I've just taught a lesson on, you know, how to create a claim for an argument.
00:45:39
Speaker
And students are working on that.
00:45:40
Speaker
They're building, they're drawing, they're talking, they're writing, and all sorts of different modalities going on.
00:45:45
Speaker
And I'm walking the room and I'm taking pictures and I'm capturing recordings.
00:45:50
Speaker
Otter is not only capturing voice to text, what I'm saying and what others are saying in the room.
00:45:55
Speaker
It's distinguishing speakers from one another and it's dropping the video and the pictures and audio into that larger transcript as I go.
00:46:06
Speaker
And then it codes all of that data for me.
00:46:09
Speaker
the minute that I hit end on the recording.
00:46:12
Speaker
So for me, that is the way that AI is enabling me and it's acting as an assistant inside of a really complex process where I'm capturing a ton of qualitative data and it would not be sustainable.
00:46:29
Speaker
If I was still teaching eighth grade and I had 130 students that I'm trying to do this with every day inside of 27 to 42 minute classes, which is what I had,
00:46:39
Speaker
Without a tool like that and the power of AI, I'm not sure I could move in that direction.
00:46:45
Speaker
And that's a totally realistic.
00:46:47
Speaker
So when people are saying, well, teachers should be doing it differently.
00:46:52
Speaker
No, this was absolutely impossible unless you had really small class sizes, really long classes, and some good assistants in the room helping you to document.
00:47:04
Speaker
And so I think the future is really bright here with these tools.
00:47:11
Speaker
Thank you again for listening to our podcast at Human Restoration Project.
Conclusion and Call to Action
00:47:15
Speaker
I hope this conversation leaves you inspired and ready to start making change.
00:47:18
Speaker
If you enjoyed listening, please consider leaving us a review on your favorite podcast player.
00:47:23
Speaker
Plus, find a whole host of free resources, writings, and other podcasts all for free on our website, humanrestorationproject.org.
00:47:29
Speaker
Thank you.