Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Off The Mark: How Grades, Ratings, & Rankings Undermine Learning (But Don't Have To) w/ Jack Schneider & Ethan Hutt
14 Plays1 year ago
“Let's start with the bad news.” is how the conclusion to my guests’ book about changing grading practice begins. “No one is coming to save us. No consultant is going to sweep through and fix things for a fee. No new technology, digital, online, or otherwise, is going to change the game.” The game, of course, is school, and the currency of that game is grades.
Jack Schneider is Dwight W. Allen Distinguished Professor in the College of Education at the UMass - Amherst. He is the Executive Director of the Beyond Test Scores Project. Director of the Center for Education Policy. Co-Editor of the History of Education Quarterly, and Co-Host of the Have You Heard Podcast.
Ethan Hutt is the Gary Stuck Faculty Scholar in Education and associate professor at the UNC Chapel Hill School of Education.
Their 2023 book, Off the Mark: How Grades, Ratings, and Rankings Undermine Learning (but Don’t Have To), is a thorough, and at times frustratingly pragmatic, exploration of flawed necessity of the load bearing pillars of “real school” – grades, transcripts, and standardized tests – their origins in our nation’s history, the distorting effects they tend to have on the outcomes and goals of education, why nothing has arisen so far to replace them at scale, and why there are no magic potions: “No one is going to wake up one morning and realize that the answer was staring us in the face all along,” they remind us.
Balancing the real with the ideal, they also chart a path toward the possibility for something different, and like the grand experiment of public schooling itself, it’s something we’ll have to figure out and build together.
Recommended
Transcript
Introduction to Educational Assessment Challenges
00:00:00
Speaker
I think my great hope for this book is not that people will read it and say, great, look, we're going to go with double clickable transcripts.
00:00:08
Speaker
We're going to go with overwritable grades.
00:00:11
Speaker
We're going to restore the use value of education.
00:00:14
Speaker
That's not our hope.
00:00:16
Speaker
At least it's not my hope.
00:00:17
Speaker
My hope is that we will just make visible the water in which we swim.
00:00:23
Speaker
That's my hope.
00:00:27
Speaker
Let's start with the bad news, is how the conclusion to my guest's book about changing grading practice begins.
00:00:33
Speaker
No one is coming to save us.
00:00:34
Speaker
No consultant is going to sweep through and fix things for a fee.
00:00:38
Speaker
No new technology, digital, online, or otherwise, is going to change the game.
00:00:43
Speaker
The game, of course, is school, and the currency of that game is grades.
Historical Context and Origins of Grading Systems
00:00:48
Speaker
Jack Schneider is Dwight W. Allen, distinguished professor in the College of Education at UMass Amherst.
00:00:54
Speaker
He's the executive director of the Beyond Test Scores Project, director of the Center for Education Policy, co-editor of the History of Education Quarterly, and co-host of the Have You Heard podcast.
00:01:06
Speaker
Ethan Hutt is the Gary Stuck Faculty Scholar in Education and Associate Professor at the UNC Chapel Hill School of Education.
00:01:14
Speaker
Their 2023 book, Off the Mark, How Grades, Ratings, and Rankings Undermine Learning But Don't Have To, is a thorough and at times frustratingly pragmatic exploration of the flawed necessity of the load-bearing pillars of real school.
00:01:30
Speaker
Grades, transcripts, and standardized tests.
00:01:33
Speaker
Their origins in our nation's history.
00:01:36
Speaker
The distorting effects they tend to have on the outcomes and goals of education.
00:01:40
Speaker
Why nothing has arisen so far to replace them at scale.
00:01:44
Speaker
And why there are no magic potions.
00:01:47
Speaker
No one is going to wake up one morning and realize that the answer was staring us in the face all along, they remind us.
00:01:53
Speaker
Balancing the real with the ideal, they also chart a path toward the possibility for something different.
00:01:59
Speaker
And like the grand experiment of public schooling itself, it's something we'll have to figure out and build together.
00:02:06
Speaker
Jack and Ethan, thank you so much for taking the time to be with me today.
00:02:09
Speaker
Yeah, thanks for having us.
00:02:11
Speaker
Yeah, that was an amazing introduction.
00:02:13
Speaker
I think we're going to have to transcribe that and use it ourselves, yeah.
00:02:18
Speaker
So I figured we would just start right away with that bad news part.
00:02:21
Speaker
Let's start at the end, right?
00:02:22
Speaker
Tear the bandage off right away.
Cultural Influences and Systemic Barriers
00:02:24
Speaker
Because if I understand the book correctly, we won't be getting rid of grades, transcripts, and standardized tests for public schools at scale anytime soon.
00:02:32
Speaker
So why the heck not?
00:02:35
Speaker
So let's start there.
00:02:36
Speaker
Yeah, it's a really good question.
00:02:38
Speaker
And one of the things that I continue to joke about when we're doing interviews about the book is the fact that we probably could have sold a lot more books if we had said something different.
00:02:50
Speaker
If we had said, you know, all you need to do is follow the Hutt-Schneider four-step plan for weaning yourself off of grades, tests, and transcripts.
00:03:02
Speaker
The problem is that
00:03:05
Speaker
they are so baked into not only the infrastructure of schooling and education and credentialism, but also to the culture of schooling and American life.
00:03:19
Speaker
So let's start with the latter of those two.
00:03:22
Speaker
If you dropped your kid off at a school that didn't give grades, what are the odds that you would say, this is fantastic.
00:03:31
Speaker
I love that the school is going to focus on
00:03:34
Speaker
intrinsic motivation rather than extrinsic motivation or any of the other positive things you might say about that, right?
00:03:41
Speaker
I think most of us would say, well, how is my kid going to get into college then if there are no grades, right?
00:03:48
Speaker
If you dropped your child off at a school that had no transcripts, right?
00:03:55
Speaker
Let's imagine it's a college or university, right?
00:03:58
Speaker
Your child has already successfully passed that hurdle.
00:04:02
Speaker
You would likely wonder, well, how is my kid going to be motivated to work hard, knowing that he or she or they, they're now 18 years old, living on their own, and there is nothing holding them accountable for making good on tuition payments there.
00:04:21
Speaker
So there's the cultural piece that we need to deal with, which is reinforced by the fact that we've all been through school ourselves.
00:04:29
Speaker
So this isn't just about the culture of the present and the ways that we use grades, tests and transcripts in our lives.
00:04:36
Speaker
It's also about the fact that we have expectations about what a real school is and does.
00:04:42
Speaker
And then there's the infrastructural piece, which is that there are all kinds of decisions that are being made all the time on the basis of test results, grades, course titles, right?
00:04:56
Speaker
That's a feature of transcripts that we often ignore.
00:04:59
Speaker
We often think of transcripts merely as places where grades and test scores reside forever permanently, but
00:05:07
Speaker
There are also places where course titles live that that offers an additional venue for students to game these metrics.
00:05:17
Speaker
And that's something that I'm sure we'll get into in this conversation.
00:05:20
Speaker
But that infrastructure would be very hard to totally upend when we're talking about 13,000 school districts, 98,000 public schools, never mind, you know,
00:05:34
Speaker
20,000 or so 30,000 private schools that we could add into the mix, and 1000s of colleges and universities.
00:05:41
Speaker
So it's a major, major undertaking to do anything.
00:05:46
Speaker
And I think what we're trying to do in this book is convince people this is this is worth our time and energies.
00:05:54
Speaker
one of the things that we try to do as historians of education is try to understand, like, where did these practices come from?
00:06:00
Speaker
And what Jack is talking about, both the cultural piece and the idea that it really holds our system together, the idea that we use common metrics, the idea that we have a common way of presenting the work of school, and that works not just for parents of students, but also future audiences of those transcripts, so colleges, employers, that
00:06:24
Speaker
All of these things, and we often in the book talk about grades, tests, and transcripts as technologies, because that's what they are, that these technologies were invented for a particular purpose.
00:06:35
Speaker
And those purposes endure in our system.
Engaging Educators in Purposeful Conversations
00:06:38
Speaker
So we need to communicate, we need to motivate, we need to synchronize our system.
00:06:42
Speaker
And so as much as we would love to sell more books and just be like, well, to hell with grades,
00:06:47
Speaker
you're removing a core function of the system and you should expect that the system is going to respond.
00:06:54
Speaker
And so we'll, I'm sure, talk about it, but it's not that people have had no ideas about alternative ways, but it's whether those ideas can actually fulfill the purposes for which we first invented these things.
00:07:06
Speaker
And so part of the conversation in the book and one that we're hoping to have with educators around the country is,
00:07:12
Speaker
OK, let's think about those purposes.
00:07:13
Speaker
Let's think about what our technologies are doing.
00:07:16
Speaker
And maybe it's time to take advantage of new technologies to update some of our practices.
00:07:21
Speaker
But you're never going to get away from those sort of core functions that grades, test scores and transcripts perform.
00:07:29
Speaker
And if you if you pretend otherwise, you're just you're just selling something.
00:07:35
Speaker
Yeah, I think you all said it very well in the book over and over again, that schools, you know, the American education system, as much as it might be anathema now to kind of say this, but they're inherently conservative institutions that resist, you know, these kinds of radical changes.
00:07:50
Speaker
And we'll get to, I think, the solutions that you all offer, but really reducing the
00:07:58
Speaker
the potential for both top-down and the bottom-up reforms back to those first principles and really seeing how can we change it to help meet communication, meet the need for synchronization, and meet those other needs that grades, transcripts, test scores.
00:08:11
Speaker
help communicate really does chart a very clear path forward.
00:08:15
Speaker
But I think before we get to that forward path, let's actually look back to perhaps the origins of the book.
00:08:22
Speaker
You wrote in the introduction that the project started before COVID.
Collaboration and Comparative Education Insights
00:08:28
Speaker
I don't know if that was 2019 or just early 2020, but
00:08:33
Speaker
COVID for all of its horribleness did provide a kind of case study for how the education system responds to these drastic, dramatic changes.
00:08:42
Speaker
And perhaps we can talk about more of what we saw in that, but I just wanna know too, what led you both to start collaborating on this topic, on this project, the stuff that eventually became off the mark?
00:08:57
Speaker
Well, we're looking at you.
00:09:02
Speaker
The project started, I think,
00:09:05
Speaker
Way back when we were in graduate school together, Ethan and I shared a dismal basement office with each other.
00:09:12
Speaker
And, you know, to bring people really behind the curtain, this office was so bad that if you sat at Ethan's desk and your office mate was not sitting at his desk, that the light sensor that needed to sense movement from you in order to provide illumination,
00:09:35
Speaker
would not register that anybody was in the office.
00:09:37
Speaker
So I would often come back to our office and Ethan would be quietly working in the dark.
00:09:43
Speaker
But we have been friends and collaborators for, gosh, at this point, you know, 15 years, maybe closer to 20 years.
00:09:54
Speaker
And
00:09:56
Speaker
One of the things that we're both really interested in, and I'll defer to Ethan for a more detailed description here because he tends to be more precise in the way he talks about this stuff than I am.
00:10:09
Speaker
But one of the things that we're both really interested in
00:10:12
Speaker
is the way that behavior and action is shaped by things like rules and measurement and systems that are often based on really thin information right i just speaking only for myself here we are such interesting complex thoughtful capable life forms and yet we pin ourselves in
00:10:41
Speaker
with these very thin rules that are not able to capture the thickness of life or the richness of what we can work through in terms of human deliberation.
00:10:56
Speaker
And for me,
00:10:58
Speaker
That irony, and it's a kind of tragic irony, is really interesting.
00:11:02
Speaker
And we could look at it anywhere.
00:11:04
Speaker
But I think it's most interesting and sometimes most tragic in education because we do it at such large scale.
00:11:12
Speaker
And we do it for what we think is the best interests of 50 million young people who are in the schools.
00:11:23
Speaker
Yeah, I was just I mean, we've always I think Jack and I have always shared an interest.
00:11:27
Speaker
We had a mutual mentor in David Tyak, who always talked about the important part of the grammar of schooling.
00:11:35
Speaker
So those things in the school that we take for granted and they're just, you know, when Jack was talking earlier about like when you show up at a school, how do you know it's a school?
00:11:43
Speaker
You know, all those things.
00:11:45
Speaker
that's part of the grammar of schooling.
00:11:46
Speaker
It's what we expect.
00:11:47
Speaker
It's how we communicate about education systems.
00:11:51
Speaker
And I think we've always shared an interest in kind of thinking about, okay, the pieces of that grammar and where they came from and what they do.
00:11:58
Speaker
And I've had in my work, both with Jack and without
00:12:02
Speaker
is a particular interest in how we communicate about the work of schools to the public.
00:12:08
Speaker
So, you know, whether it's test scores and grades.
00:12:11
Speaker
And so one of the interesting things about these technologies, these assessment technologies, is that they have multiple audiences and those audiences have expanded in scope over time.
00:12:20
Speaker
And so thinking about how
00:12:23
Speaker
these practices have evolved has always just been really fascinating to me.
00:12:28
Speaker
And just, you know, we, we wrote a piece, our very first, I think, collaboration together, I think it was in 2014.
00:12:33
Speaker
We, we first wrote about the grading system and, you know, we just kept coming back to those kinds of, of questions.
Understanding and Balancing Stakeholder Needs
00:12:42
Speaker
And then obviously, you know, we've,
00:12:43
Speaker
all experienced together, no child left behind, which added another layer of this sort of communication.
00:12:49
Speaker
So thinking about those practices and again, coming from them, as you said, like from a place of like there's preserved wisdom in some of these inherited practices doesn't mean they're perfect and can't be improved.
00:13:01
Speaker
But I think especially when you look at the history of school reform,
00:13:05
Speaker
Too often we have reformers who are like, I know better.
00:13:08
Speaker
The teachers don't.
00:13:09
Speaker
Everything is broken.
00:13:10
Speaker
Let's throw it out.
00:13:11
Speaker
And, you know, my sister is a second grade teacher.
00:13:13
Speaker
My mom was a teacher.
00:13:15
Speaker
When you talk to teachers, they have a lot of understanding and knowledge about the system.
00:13:19
Speaker
And so this idea that schools are conservative or that reform should approach things in a conservative way.
00:13:26
Speaker
What we're talking about a lot of times is preserving the system.
00:13:29
Speaker
both the shared experience of folks, but also the knowledge when teachers look at a lot of reforms, they go, that's not going to work.
00:13:35
Speaker
You can make me try to do this, or maybe I will just not do it, but it's not going to work.
00:13:39
Speaker
And so thinking and treating the system with care rather than just taking a wrecking ball to it, which may speak to our kind of like our deeper, you know, let's burn it all down.
00:13:50
Speaker
But practically speaking, those reforms are not built to last, no.
00:13:56
Speaker
That's what I think I mentioned in the intro, right, was that frustrating pragmatism as someone who kind of considers them, you know, myself as a when I was in the classroom practicing grades, really leaning into ways that I can diminish, you know, the salience, the importance of grades and grading, really changing reporting practices in my own practice.
00:14:17
Speaker
And now, you know, working at that big systems level to try and say, how can we
00:14:21
Speaker
improve the system both for students, but also make it workable for teachers and community and parents alike.
00:14:26
Speaker
Like trying to triangulate all of those needs can't be done simply by changing one part of that equation, right?
00:14:33
Speaker
You have to get all of those folks at the table and ask what is going to work to meet the needs of everybody.
00:14:41
Speaker
And really, you know, primarily the work that we that my organization does is focus groups with kids, with community members.
00:14:47
Speaker
And when you actually sit those two groups down and look at what they say about the changes they want the system to be, kids and parents predominantly like agree on the ways in which that system could change.
00:14:58
Speaker
So there's a lot more, I think, collaboration there than, again, leaders who are kind of at the top of the system trying to
00:15:08
Speaker
move those top-down reforms without getting buy-in from teachers or teachers not getting buy-in from students or schools not getting buy-in from parents, right?
00:15:16
Speaker
We've kind of lost that community aspect to it.
00:15:20
Speaker
I really want to know then in the process of researching
00:15:24
Speaker
And writing the book, which sounds like it'd been going on for basically, this is the culmination of a decade of work on this, a 15-year collaboration.
00:15:33
Speaker
What about that process of research and writing this book, either challenged, affirmed, or changed your perspectives on grades, transcripts, testing, any one of those kinds of concepts that you guys look at?
00:15:46
Speaker
So it's interesting.
00:15:47
Speaker
I mean, there's so much that you learn.
00:15:49
Speaker
I think the very first thing that I learned was, you know, is sort of where you started, where, you know, you kind of are led to believe, you know, when you enter like, oh, American public education is bad.
00:16:02
Speaker
And, you know, Finland has the answer and Germany has the answer.
00:16:05
Speaker
And, you know, Korea has the answer and the Soviets have the answer.
00:16:08
Speaker
And like the very first thing you sort of realize as you look at other practices, and especially if you take seriously the idea that,
00:16:15
Speaker
you know, it's that there's a cultural development element to these practices that there's no secret sauce.
00:16:22
Speaker
And I mean, it's like you can you can take some exotic, you know, practice.
00:16:27
Speaker
It's like taking a plant out of like, you know, the rainforest and then moving it to like the desert of L.A.
00:16:31
Speaker
It's like it's like there's so much that goes into these.
00:16:35
Speaker
It's not a it's not a technical problem in the sense of like, oh, you just you just grab something off the shelf or grab some secret practice.
00:16:41
Speaker
I've spent time abroad and I figured this out.
00:16:44
Speaker
And once you sort of give up the idea that there is some holy grail out there of practices, and as you say, focus on like the sort of human element, like what are the problems that these folks were trying to solve over time?
00:16:58
Speaker
You know, you just realize how embedded they are.
00:17:00
Speaker
Like one of the things that never really had occurred to me.
00:17:05
Speaker
in a real concrete sense until I got into it was like how there really is historically a real symbiotic relationship between this idea of local control, the idea that we want communities to be able to make decisions for themselves and have flexibility in talking about what practices make sense, what courses make sense, what trajectory for our students makes sense.
00:17:26
Speaker
And this need, because we live in a very large country that has a decentralized system, and I think most of us like on a regular basis like that, that we have a school board and we have teachers that have flexibility.
00:17:40
Speaker
that that almost requires somewhere in the system some standardization, some points of synchronization, as we call it in the book, that allows people to move from that decentralized local practice to some higher level.
00:17:56
Speaker
And you see this with transcripts and this idea that we need a common currency to record like, well, how much learning is learning?
00:18:03
Speaker
We need a unit.
00:18:04
Speaker
And we give time, you know, pride of place there.
00:18:07
Speaker
But same with, you know, things like the SAT.
00:18:10
Speaker
I've done work on the GED.
00:18:12
Speaker
Like having these tests that kind of sit on top and allow for flex in the system, those practices come together.
00:18:20
Speaker
So we often talk about standardized tests and grades as separate.
00:18:23
Speaker
And one of the things that we really liked about the book, because we had done that in our own writing, having them separate, was putting them together and actually say like,
00:18:31
Speaker
You can mess with these, but often the other one is picking up the slack.
00:18:35
Speaker
So if you want to get rid of the SAT, you're putting more weight on grades.
00:18:40
Speaker
And if you want to get rid of grades, like some countries do, no one looks at transcripts, you're going to put all the weight on a test.
00:18:47
Speaker
So thinking about how these are actually sort of in conversation, they're linked in the system, was one of the big revelations is that we have to think about these as a whole system set of practices.
00:18:59
Speaker
and not just like tinker with one practice or the other.
00:19:03
Speaker
Yeah, Ethan got into this in his response there.
00:19:07
Speaker
But one of the things that I learned from him along the way was to be a cold-hearted realist about this and not just be a dreamer.
Recognizing Utility and Problems of Grades
00:19:20
Speaker
I am a former classroom teacher.
00:19:22
Speaker
And I think it is my inclination to try to take big swings at things.
00:19:30
Speaker
And I think had I tried to write this book,
00:19:33
Speaker
or do any of this work.
00:19:35
Speaker
without Ethan, I might have verged into, you know, not snake oil sales, but, you know, I think dreaminess, right?
00:19:46
Speaker
Like, I'm good at identifying all of the problems that exist.
00:19:53
Speaker
I can just rattle off all of the ways that grades distort motivation and undermine the kind of communication that educators have
00:20:04
Speaker
in their hearts want to do with young people.
00:20:08
Speaker
I can tell you all of the ways that standardized testing, whether it is run by the state or an external entity like the college board, is designed in a manner that fails to incentivize real authentic learning, undermines the practices of educators that are responsive to the real needs of young people.
00:20:28
Speaker
I can go on this all day, but
00:20:32
Speaker
What I don't naturally do is think about the usefulness of things like standardized tests or the ways in which grades actually do serve a real important purpose, including for equity, which is often a critique that I will level against grades, but
00:20:49
Speaker
You know, one of the things that I learned from Ethan in doing this project was that a grade has just enough vagueness to it, right?
00:21:00
Speaker
That it can allow for important kinds of slippage.
00:21:06
Speaker
And what I mean by that is that we take for granted that a course title has a particular meaning.
00:21:14
Speaker
and that a grade in that course has a particular meaning.
00:21:18
Speaker
And that allows for formal equity in our educational system in a way that I think it is hard to imagine existing if we said, well, let's allow people to just use their own judgment with no formal guidance, no restrictions, no rules here in interpreting.
00:21:41
Speaker
what it means that a student in a high poverty community from a historically marginalized group took a particular math course and has a particular kind of recommendation from their teacher, right?
00:21:56
Speaker
And so although I, earlier in our conversation,
00:22:01
Speaker
you know, made a little speech about, you know, the thinness of information and the way we pin ourselves in with rules.
00:22:07
Speaker
You know, I think I'm speaking to my, my natural interests there.
00:22:11
Speaker
But one of the things that I learned in this project is rules also at the same time are really important.
00:22:18
Speaker
And so one of the things that I think we do in this book is try to help people think in clear ways.
00:22:27
Speaker
Like we have a lot of faith in educators and in families and some faith in policymakers, but to try to structure for them a way of thinking through when are rules useful, right?
00:22:42
Speaker
When is data productive, right?
00:22:44
Speaker
When are these kinds of assessment technologies really important?
00:22:49
Speaker
And then where do we need to really blur the edges?
00:22:53
Speaker
Where do we need to remove some of the rules?
00:22:56
Speaker
Where do we need to pull the guardrails a bit wider?
00:23:01
Speaker
I will say it was definitely interesting to read because I felt that tension, that push and pull between, you know, that pragmatic realism and then that, but that, that, the pull of the, the push of the idealism there too.
00:23:13
Speaker
And I think as an idealist or as a dreamer, you know, myself, I, I found myself having to sit in that tension, right.
00:23:21
Speaker
And, and grapple with that reality of it.
00:23:23
Speaker
And,
00:23:24
Speaker
Again, I appreciated how the book just brings it back to the first principles, to the history of how these practices even emerged and just...
00:23:33
Speaker
kind of distilled them down into their most component parts to say, look, we can kind of change within this range of options here to diminish distortions or to increase equity.
00:23:44
Speaker
And basically we can fiddle with the sliders, but these three ideas have emerged to serve a particular role.
00:23:52
Speaker
And I think kind of shifting the conversation perhaps towards those potential solutions where you guys found them, either internationally,
00:24:02
Speaker
There's a whole section in the book where you spoke with international teachers abroad and spoiler alert for anyone out there.
00:24:10
Speaker
Right.
00:24:10
Speaker
They didn't find magic potions out there either in any country that they talked about.
00:24:16
Speaker
But what you know, what does the successful reform of these practices, be they grades or transcripts or the standardized testing components?
00:24:25
Speaker
What does that look like in your view?
00:24:27
Speaker
Like what examples did you find abroad, here at home, contemporary ones in the mastery transcript or historical examples that you pulled?
00:24:34
Speaker
You guys draw from such a wide well of disciplinary knowledge, right?
Innovative Approaches to Transcripts and Grading
00:24:41
Speaker
And then how do we realign the goals of grading, assessment, and reporting with the goals of learning?
00:24:48
Speaker
So I'll give my favorite one.
00:24:50
Speaker
And then Jack, I know has a different favorite one from the book.
00:24:54
Speaker
So, and first I'll say, any idealists out there who just need someone to ruin their day, I'm happy to get on the phone and just tell you, I've often joked that I should just start a consulting firm that's just called, dude, that's not gonna work.
00:25:07
Speaker
And I'll just explain to you
00:25:09
Speaker
Why is it not going to work?
00:25:11
Speaker
And you can take it early.
00:25:13
Speaker
Please don't listen to me.
00:25:13
Speaker
That's no fun.
00:25:15
Speaker
What's nice about writing with someone who has a little bit of a different temperament is like you do get that push and pull.
00:25:21
Speaker
And I am glad that it came out in the book as being a real conversation between the two of us.
00:25:27
Speaker
And we've had people say, so which sections did you write and which section?
00:25:30
Speaker
And it's like, no, it's really interwoven in that way.
00:25:34
Speaker
So I'm glad it comes to you.
00:25:36
Speaker
So I'll give a good example of something that it feels like
00:25:39
Speaker
the history of the practice helps inform, you know, kind of a new way of thinking about it that recognizes the value and recognizes the the the challenge, but actually moves us forward.
00:25:51
Speaker
So, you know, historically, if we think about a transcript,
00:25:57
Speaker
there were very concrete, practical, physical reasons why you needed a transcript to be short.
00:26:03
Speaker
It needed to have, it needed to distill all the richness of an education to a single sheet of paper because in the world before computers, like those transcripts were housed in physical spaces, in filing cabinets,
00:26:18
Speaker
And you needed the school to keep that for a very long time.
00:26:21
Speaker
So there's a real physical space reason why it made sense for schools to try to distill and have a real economy of communication, economy of record keeping around student transcripts and student, you know, student performance.
00:26:38
Speaker
But those space constraints don't really apply anymore.
00:26:42
Speaker
I mean, my transcript takes up a few bytes of space on a computer that basically has, with the cloud, infinite space.
00:26:48
Speaker
So one of the ideas that we talk about is that if one of the challenges of the transcript is that the information is so thin, I can bug Jack to give me a slightly higher grade, and he can record an A instead of an A minus, or let's be honest, an A minus instead of a B plus, then
00:27:06
Speaker
we can, you know, then I know that, you know, mission accomplished.
00:27:10
Speaker
No one's ever going to see that essay and no one is ever going to be able to decide whether my final project was or wasn't an A minus or B plus or whatever.
00:27:18
Speaker
It's like once it's on the transcript, like that's the end of the game.
00:27:22
Speaker
But
00:27:23
Speaker
And for reasons like I explained, like the lack of space means that there's no work behind that grade.
00:27:30
Speaker
And so one of the things we talk about in the book, one of the things that seems very within reach is if we made transcripts what we call double clickable.
00:27:37
Speaker
So if you could, for instance, not only deliver your transcript, but behind the grade of that transcript existed any amount of work that reflected the work behind that grade,
00:27:51
Speaker
We argue, and I think it feels right to me, that that helps focus the learner and the whole enterprise of education around what is the learning that is associated with that grade.
00:28:05
Speaker
By reattaching the work and the work product to the grade that sits on the transcript, you allow people to focus on not just the letter, but what's actually learned.
00:28:16
Speaker
And you can tell someone, a student, say, look,
00:28:20
Speaker
these are not going to be detached forever.
00:28:22
Speaker
Like someone's going to look and you should actually want them to look because then they can see all the things that you did and all the things that you accomplished.
00:28:30
Speaker
And maybe they even see the whole process of your accomplishment.
00:28:33
Speaker
So by making a transcript, by thinking about like, okay, the limitations no longer apply, we can recenter learning.
00:28:39
Speaker
We can recenter the actual product of learning rather than just have students motivated by that, by that letter on that transcript.
00:28:49
Speaker
Yeah, Ethan allowed me to go with my favorite here.
00:28:55
Speaker
So I am really enamored by the idea of an overwritable grade.
00:29:03
Speaker
So I'll start with the buzz phrase there, where we are referring to basically the digital capacity to replace old information with new information.
00:29:16
Speaker
And I'll back into an explanation of how we arrived at that as a possible way of tinkering towards reform.
00:29:25
Speaker
So one of the things that we talk about in the book is the fact that these assessment technologies arose to serve different purposes.
00:29:35
Speaker
And those purposes can be sort of shorthand conveyed as the purpose of communication.
00:29:46
Speaker
And there are actually two distances of communication that we talk about in the book, a short-haul communication, an educator trying to communicate with a student and or their family, and a long-haul communication,
00:29:59
Speaker
So the school, the educator, and sometimes even the student themselves having an interest in this, communicating with distances, blah.
00:30:11
Speaker
And long haul communication where an educator or sometimes the students themselves have an interest in communicating with audiences at a distance that is either geographic or temporal, right?
00:30:26
Speaker
Communicating with people who are really far away or who are in the future.
00:30:32
Speaker
So that's communication.
00:30:33
Speaker
Then there's also motivation.
00:30:35
Speaker
That one's pretty obvious, right?
00:30:36
Speaker
The use of assessment technologies as a way to try to get young people to work hard and show up.
00:30:43
Speaker
And we'll just name the fact that, you know, we have compulsory education laws for a reason, right?
00:30:48
Speaker
There are lots of other things to do in this life than go to school.
00:30:54
Speaker
and then third is synchronization to try to stitch together the component parts of a very large cumbersome and mostly decentralized system well let's just talk about the communication function that grades served in their origin right they were serving a short-haul communication function they really were an effort to communicate in shorthand
00:31:20
Speaker
with students' families.
00:31:22
Speaker
And that was related to motivation very much, right?
00:31:25
Speaker
Because the idea was maybe we can get these young people to work a little harder if their families know that they aren't applying themselves in school.
00:31:34
Speaker
And maybe we can get them to continue to work hard if their families praise them for all the good work that they're doing.
00:31:40
Speaker
So these purposes aren't always necessarily disconnected from each other, although they do often work across purposes.
00:31:47
Speaker
And so let's just consider how the fact that grades over time, so as we move from the early to mid
00:31:56
Speaker
19th century into the 20th century and into the mid 20th century, we can see the grades are increasingly being used by the time we arrive at the mid 20th century as a way of communicating with much more distant audiences, right?
00:32:13
Speaker
By the mid 20th century, high school enrollment is pretty close to universal.
00:32:18
Speaker
we are moving closer to universal high school graduation.
00:32:22
Speaker
We still aren't there yet.
00:32:23
Speaker
We may never arrive there, right?
00:32:25
Speaker
But we get close enough that we get far more students attending college as a way of trying to distinguish themselves.
00:32:31
Speaker
And we see very real labor market benefits for students who get high school diplomas, who have good grades as a result of hard work.
00:32:43
Speaker
in high school and who go to college,
Impact of Grades on Student Motivation
00:32:46
Speaker
right?
00:32:46
Speaker
And very real labor market returns for students who graduate from college and get good grades.
00:32:52
Speaker
And so we can see that students develop this very real interest in getting good grades in order to reap the rewards of what is essentially a credentials market there.
00:33:07
Speaker
We've got two different forms of communication there.
00:33:10
Speaker
And if we think about the ways that they are not really designed to work together, we can begin to see problems.
00:33:19
Speaker
So the idea that grades were originally intended to serve a short-haul communication function and then eventually get used to serve this long-haul communication function leads to a problem where educators will often want to say, hey, you don't get this.
00:33:35
Speaker
right?
00:33:36
Speaker
I'm trying to communicate to you with this grade that you do not fully understand this material.
00:33:40
Speaker
And I would actually like your family to know this because lo and behold, families care about the young people who live with them and want the best for them and often can intervene.
00:33:51
Speaker
And we could have a whole separate conversation about how we end up in this country, mostly offshoring the responsibility of giving extra assistance to families and the free market rather than doing it systematically.
00:34:04
Speaker
which is something that grades could actually serve a useful function in.
00:34:08
Speaker
But we can see that if you're trying to send this message to students and their families, hey, you don't get it, you're also doing something at the exact same time, which is ruining that student's future, right?
00:34:20
Speaker
That's how the student is going to interpret it.
00:34:23
Speaker
is, hey, you thought that you were sending me a message by giving me a C in algebra, but do you know what you actually did?
00:34:32
Speaker
You actually put a black mark on my record.
00:34:36
Speaker
And this can be not just a problem for ambitious students who want to get into highly selective colleges and universities, who want to have pristine transcripts, you know, and have more than a 4.0 grade point average.
00:34:51
Speaker
This can be devastating when we're talking about students for whom school isn't a natural fit, for whom there are headwinds rather than tailwinds.
00:35:02
Speaker
These black marks on their records pile up.
00:35:05
Speaker
And if the incentive is, hey, go to school and work hard because that's going to open up social and economic opportunities for you.
00:35:13
Speaker
Well, if they see, hey, listen, I've already got a bad enough academic record that I'm not going to benefit from this, right?
00:35:21
Speaker
Then that further undermines the motivation of those students.
00:35:26
Speaker
So that's a long wind up to a way of dealing with this problem, which is to say, well, if you made it possible to overwrite the student's grade once the student gained the relevant knowledge or skill,
00:35:42
Speaker
You could actually have it both ways.
00:35:44
Speaker
You could actually communicate to the student and their family, hey, you don't get it right now, right?
00:35:49
Speaker
Right now, you've got a C in algebra or geometry or English literature or physical education, right?
00:35:57
Speaker
You've got a C.
00:35:59
Speaker
But that's not where this story needs to end.
00:36:02
Speaker
And the example that I always give, right, Ethan's going to get sick of this example, is that none of us knew how to ride a bike once upon a time.
00:36:10
Speaker
Nobody came out of the womb on a bicycle, as far as I know, right?
00:36:14
Speaker
But if we had been graded,
00:36:18
Speaker
at the point in time where somebody had arbitrarily said, let's see if you can do it, many of us would have failed.
00:36:25
Speaker
And we would have had to carry around on our transcripts an F in bike riding, despite the fact that the vast majority of us later learned that skill, right?
00:36:33
Speaker
Why carry around outdated information?
00:36:35
Speaker
Maybe you did fail algebra, and maybe you then learned that skill.
00:36:41
Speaker
Why carry the F forever on a transcript instead of being able to overwrite that information
00:36:47
Speaker
with more accurate information that actually is going to better serve the purpose of communicating to future audiences what you know and can do.
00:36:57
Speaker
And I think that's another thing that I appreciated about the book was in the sense that you explore historical examples of narrative reports and of portfolio systems and all of these other kind of one size fits all replacements,
Proposing Effective Reforms in Assessment
00:37:14
Speaker
right?
00:37:14
Speaker
That were meant to replace, upend a revolution in grading, testing, reporting, and all these things.
00:37:22
Speaker
And really, I think a message that I took away from the book was certainly that
00:37:26
Speaker
you know, these hybrid models where we're kind of Frankensteining together, the best purposes of every kind of system are going to be the things that are ultimately going to make any successful grading, reporting, testing reform work in the long run is not trying to replace one, the flaws of one, one size fits all with the flaws of another one size fits all, but really trying to take the good where we can find it and
00:37:52
Speaker
wherever that might come from.
00:37:53
Speaker
And again, that that's not a huge appeal to my dreamer idealism, but gosh, that makes a lot of pragmatic sense as somebody who has to work within the system, who has kids in the system, too, who has to attend to systemic coherence.
00:38:06
Speaker
So, again, I appreciated that long view, both historically and then, you know, in terms of depth of our system and the need to communicate from K-12 and beyond to employers to to the whole nine yards.
00:38:20
Speaker
I think then one thing kind of wrapping up the conversation here is just to get from both of you kind of a sense of what do you think is really missing in the public conversation around grades, transcripts, testing, the focus of the book and everything.
00:38:37
Speaker
What would be your goal about how we can speak about these things differently or how we can enrich the public conversation?
00:38:45
Speaker
Short of buying your book, of course, but then how can we speak to people differently about these distortions and the problems and the benefits and the equity and the need to communicate?
00:38:59
Speaker
I'm going to beat Ethan to the buzzer here just so I can respond to the thing that I think we both want to respond to before the question, which was your comment there.
00:39:09
Speaker
And both of us owe a debt of gratitude to David Tyak and Larry Cuban because totally embedded.
00:39:18
Speaker
in the way that you were just describing our project are some ideas that entirely come from them.
00:39:27
Speaker
And this idea of Frankensteining or hybridizing, that's a Tayac and Cuban idea.
00:39:33
Speaker
And the idea that we can't just wash away the past, that
00:39:39
Speaker
Every layer of educational policy, every layer of the culture of schooling builds upon the previous layers.
00:39:48
Speaker
That's a Tayek and Cuban idea.
00:39:51
Speaker
And even our dedication of the book, the book is dedicated to our students, is a ripoff of David and Larry.
00:39:59
Speaker
which is not to say that the rest of the book is, but we would encourage people to read side by side our book with Tinkering Toward Utopia by David and Larry.
00:40:12
Speaker
So now I'll actually answer the question and try not to be so long-winded and then kick it over to Ethan.
00:40:19
Speaker
But
00:40:20
Speaker
I think my great hope for this book is not that people will read it and say, great, look, we're going to go with double clickable transcripts.
00:40:28
Speaker
We're going to go with overwritable grades.
00:40:31
Speaker
We're going to restore the use value of education.
00:40:34
Speaker
That's not our hope.
00:40:36
Speaker
At least it's not my hope.
00:40:37
Speaker
My hope is that we will just make visible the water in which we swim.
00:40:43
Speaker
That's my hope, because I know that I speak for both of us, that we have tremendous confidence in the three and a half million educators who are at work every day in this country.
00:40:54
Speaker
We have tremendous confidence in the ability of the 50 million students who are in our public schools to speak up for themselves, including in the kind of work that you guys do.
00:41:05
Speaker
And we have a lot of faith in families who really want the best for young people.
00:41:10
Speaker
But
00:41:12
Speaker
But we inhabit an educational policy space and an educational culture that makes invisible practices that are totally ordinary to us.
00:41:25
Speaker
Not ordinary because they don't have any negative unintended consequences or don't matter, but ordinary because we completely expect them.
00:41:35
Speaker
We talked earlier about quote unquote real school and the grammar of schooling.
00:41:39
Speaker
It would be odd to us if we didn't see grades, tests, and transcripts.
00:41:43
Speaker
And in fact, we totally expect the unintended consequences as well.
00:41:48
Speaker
We say things like, well, tests are just a part of life.
00:41:52
Speaker
Or we will say out of one side of our mouths, hey, you should be really focused on learning this material.
00:41:58
Speaker
This material really matters.
00:41:59
Speaker
And then out of the other sides of our mouths, we'll say things like, hey, you have to get a better grade in this course.
00:42:04
Speaker
You don't want to have this on your transcript.
00:42:07
Speaker
And so what what my big hope is, is that people will read this book and think about grades, tests and transcripts not as ordinary features of real schools, but as technologies that we continue to choose to use often inappropriately every single day in school, 180 days a year.
Empowering Educators and Families Through Knowledge
00:42:33
Speaker
for all 13 years of the K through 12 experience and then into higher ed for students who go to four or, you know, in our case, you know, 10 more years after that.
00:42:46
Speaker
So if we can see what is presently invisible to us, then we can talk about it.
00:42:51
Speaker
And if we can talk about it, then we can act.
00:42:55
Speaker
Yeah, I'm going to just echo.
00:42:56
Speaker
I mean, I, the single, I'm not an ideas person.
00:42:59
Speaker
I'm a, I'm a, I'm a grumpy person.
00:43:02
Speaker
I'm not going to work, but I do, I do think that it strikes me when, when I listen to people, um,
00:43:10
Speaker
like struggle against our grading system.
00:43:13
Speaker
And they, you know, we saw this in the pandemic where lots of things were disrupted and people were having really difficult conversations in the public about what was fair and what was possible and what would happen and the long-term consequences.
00:43:27
Speaker
And could we set this aside and should we just give everyone an A and what would happen?
00:43:30
Speaker
Like,
00:43:31
Speaker
It struck me as that what we lacked was a real common language for having these conversations.
00:43:38
Speaker
And so, you know, especially I think the project started before the pandemic.
00:43:43
Speaker
But what the pandemic, I think, really added to the book was really it foregrounded for me and Jack that we needed to give people one of the best things we could do is give people a language to talk about.
00:43:56
Speaker
all of these different functions that grades serve.
00:43:58
Speaker
And then they're underlying, like where they come from in the system.
00:44:02
Speaker
And once we provided that, and if you look at the last chapter, it's, we really imagined the last chapter being like, we, we thought like, well, maybe my sister and her colleagues, um, uh, it,
00:44:16
Speaker
you know, would sit down with the book and say, OK, let's think about our practices.
00:44:20
Speaker
And now that we have this language, maybe we can talk about, OK, what about this and what about this?
00:44:24
Speaker
Because we really do believe in the strength of a plural approach to learning and that there isn't a one size fits all.
00:44:33
Speaker
And so providing a language that people could be on the same page and talking about it, but arrive at different conclusions about what makes sense for them and their students or their communities.
00:44:44
Speaker
That to me was the big step that we can take forward because I will
Conclusion: Balancing Idealism and Realism
00:44:49
Speaker
not.
00:44:49
Speaker
I mean, we have solutions.
00:44:50
Speaker
We pitch some to your listeners.
00:44:53
Speaker
But it would be totally fine with me if everyone decided that those were bad ideas.
00:44:57
Speaker
If they said, OK, actually, what we need is not the double clickable.
00:45:02
Speaker
What we need is...
00:45:03
Speaker
this and this and we can serve those functions.
00:45:06
Speaker
So providing some clarity and hopefully the history kind of comes through and people can see why we got to where we were.
00:45:13
Speaker
It wasn't just bad actors.
00:45:15
Speaker
It wasn't just random.
00:45:18
Speaker
It wasn't just some like factory model that someone, you know, these are these are learned things that have accreted over time.
00:45:24
Speaker
So that's the big thing that I hope people will take from the book, whether they like our solutions or not.
00:45:29
Speaker
And that's the whole thing I hope that will endure and sort of animate or help illuminate conversations in the future.
00:45:35
Speaker
Right.
00:45:36
Speaker
None of these systems fell from the sky in a single piece to be implemented as is, right?
00:45:40
Speaker
They're the result of, you know, that both the social construction, the historical construction, but I think to emphasize that construction part recognizes how it's situated in the past and in a collective reality, but also kind of recognizes the potential and possibility that, hey, we can do something different to help meet our needs and the needs of our kids here in the present or perhaps in
00:46:05
Speaker
in the future too.
00:46:06
Speaker
So again, I just, I really appreciate that, that, that balance of that realism, that idealism, that, that, that pragmatism, I think for me was, it was a huge appeal of the book.
00:46:17
Speaker
So the book of course is off the mark, how grades, ratings, and rankings undermine learning, but don't have to from Jack and Ethan.
00:46:26
Speaker
Thank you guys both so much for joining me today.
00:46:29
Speaker
Thank you for having us.
00:46:30
Speaker
Yeah.
00:46:30
Speaker
Thanks for having us.
00:46:30
Speaker
This was fun.
00:46:34
Speaker
Thank you again for listening to our podcast at Human Restoration Project.
00:46:37
Speaker
I hope this conversation leaves you inspired and ready to start making change.
00:46:41
Speaker
If you enjoyed listening, please consider leaving us a review on your favorite podcast player.
00:46:45
Speaker
Plus, find a whole host of free resources, writings, and other podcasts all for free on our website, humanrestorationproject.org.
00:46:52
Speaker
Thank you.