Transcript
Speaker: Hi there.
Speaker: So my name is Nick Covington.
Speaker: I'm from the Human Restoration Project.
Speaker: And we're here to talk today about the NAEP results.
Speaker: So the results of the National Assessment of Educational Progress, or NAEP, were released yesterday, September 1st, prompting a New York Times headline that read, The Pandemic Erased Two Decades of Progress in Math and Reading.
Speaker: The 74 headline added two decades of growth wiped out by two years of pandemic.
Speaker: Peter Green, an education policy watcher, called it NAEP pearl clutching day.
Speaker: And I myself even tweeted a lot about this.
Speaker: You know, with the release of the pandemic NAEP scores, we're about to have the worst cycle of education discourse imaginable.
Speaker: And man, did that ring true.
Speaker: So yeah.
Speaker: Everyone seems to run to their corners to abolish teachers unions, attack the remote and hybrid learning precautions that we took, mask mandates, and just relitigating every pandemic issue imaginable.
Speaker: The results even brought out the usual resident experts in everything, like Matt Iglesias, who called the scores immediately.
Speaker: a short-term L for the left that was more supportive of closure.
Speaker: So while everyone online is jumping to conclusions, we thought it would be important to help provide some context, step back and take
Speaker: take inventory of the data, the claims, the headlines, and provide some context and forecast the next steps.
Speaker: So what, if anything, could or should we do in response to that report?
Speaker: So I reached out to author and educator John Warner, whose intuition I tend to trust on this kind of thing.
Speaker: John's the author of several books, Why They Can't Write, The Writer's Practice, and Sustainable Resilient Free, The Future of Public Higher Education, released in 2020.
Speaker: So thanks, John, for making the time to turn around and talk with me about that today.
Speaker: Oh, my pleasure.
Speaker: Good to see you and try to make some sense of this stuff that's going on.
Speaker: Yeah, let's get right into it.
Speaker: So let's just start with what the results say and what do they mean?
Speaker: And maybe we can compare that to the headlines as well.
Speaker: What do the results say?
Speaker: What should we make of them?
Speaker: So in contrast to the headlines, which frame this around this idea that somehow learning the nine-year-olds, again, we've got to remember what we're talking about.
Speaker: We're talking about nine-year-olds
Speaker: who took a single test for reading and math back sometime in the early part of the year, have scored at a level that is consistent with student scores on average from 20 years ago.
Speaker: The history of the NAEP test has been a kind of relatively steady growth and increase in scores.
Speaker: So when we say it's equivalent to 20 years ago, it's
Speaker: significantly lower than students who took it the last time it was administered, which was 2020, and therefore on par with students who took it in, more like students who took it in 2002.
Speaker: The headlines erasing two decades of progress or two decades of growth are wiped out are absurd.
Speaker: that these were allowed to be published in major national publications should be to their everlasting shame because that's not what this data says.
Speaker: What this data says is at one particular time and place, nine-year-olds scored this on this exam for reading and math.
Speaker: We can now take a step back and ask, so what are the factors that have gone into this score that is significantly lower than the last time it was taken, that is on par with 20 years ago prior to what has been a sort of gradual increase?
Speaker: But the idea that suddenly our nation's nine-year-olds are 20 years behind where they should be, it's nuts.
Speaker: And even just thinking...
Speaker: Let's assume that's true, right?
Speaker: That somehow our nation's nine-year-olds have been getting steadily smarter over the last 20 years.
Speaker: We're learning steadily more over the last 20 years.
Speaker: How are our 28, 29, 30-year-olds doing today?
Speaker: Just fine, right?
Speaker: The notion that these scores are dispositive, that they...
Speaker: carry some sort of inevitable predictive weight for a future 20 years from now, 10 years from now, five years from now, is a total abuse and misuse of what a single test and a single data point means.
Speaker: It doesn't mean it's meaningless, but it does not mean what these screaming headlines say.
Speaker: And I think absent from those singular pieces of data that are going around now is all of the context of the last two years to go alongside that too.
Speaker: Something that you've commented on and I've commented on as well is, I mean, missing from reducing the experience of the last two years into two singular points, and particularly just for reading and math, disregarding every other
Speaker: subject that kids would study in school, but kind of just disregarding the project of the last two years, which might not have been to keep raising math and reading scores, but instead, you know, to turn schools into safer learning environments that
Speaker: adults and students alike can inhabit without getting each other sick, risking bringing infection home to others, you know, scrambling for schools to be able to provide the same level of services that schools have provided in terms of free meals to students and, and, um,
Speaker: And obviously, you know, child care then for working parents as well.
Speaker: So there was just a seismic cultural shift across the entire board.
Speaker: And one of the reasons why I think, I mean, the NAEP is called, you know, the nation's report card.
Speaker: It's the gold standard for these standardized assessments is even though the world might change around it, you know, NAEP always stays the same.
Speaker: And yet here we have this assessment result that seems incongruous with our experience of the last two years.
Speaker: I was just looking back at a Gallup survey from
Speaker: earlier this summer that said K-12 workers are the most burned out employees in the United States.
Speaker: Higher burnout rates than medical workers, than anybody else.
Speaker: College faculty are on that list as well.
Speaker: So when you look at a five-point decline in math or reading scores,
Speaker: alongside teachers are the most burned out employees in the entire country, you say, well, what were we doing?
Speaker: You know, if we were just sitting at home doing nothing, surely we wouldn't be as burned out as we were.
Speaker: So why does that framing matter?
Speaker: You know, like how could, what could a more meaningful, I would even say like conscientious, you know, conscientious reporting of these results do what's missing from this discourse that we're having online about this?
Speaker: Well, one of the things we should recognize is as the nation's report card, as the snapshot of something that, of a test on a single day in a single context of a single time, is disruption has happened.
Speaker: And this is now reflected in the data.
Speaker: Whatever schooling was in whatever form it took under whatever circumstances was disrupted by a global pandemic.
Speaker: And so not only school was disrupted, the entire lives of everybody in the country and the world was disrupted.
Speaker: So the notion that we could have avoided this with some kind of different school or different choices, going back in person sooner or anything like this, it's not reflected in any of the data.
Speaker: In fact, in some of the underlying data they release, there's, there's, uh,
Speaker: indications that none of that mattered, right?
Speaker: Like in reading scores, students who are in city districts, their reading scores were actually flat, the only metric that didn't go down.
Speaker: And these were schools that were much more likely to be closed for longer.
Speaker: Regional differences don't tell us much of anything in terms of schools in the South, which were more likely to go back to school in person sooner.
Speaker: had decreases equivalent to, and in some cases greater than, schools in the Northeast, which were likely to have gone back in person later.
Speaker: So what we have really is a picture like a big thing happened.
Speaker: And this big thing has affected how students perform on this test.
Speaker: And it could be any number of reasons.
Speaker: It could be something like actual specific test prep is the thing that was sacrificed.
Speaker: And if that was the case, that would be a really interesting finding.
Speaker: It would tell us that these sorts of tests, when they're prepped for, maybe don't tell us all that much about what students are learning.
Speaker: I don't know how big a factor that is or if it's a factor at all.
Speaker: It could be that students were grieving the loss of a caretaker.
Speaker: It could mean that students who were, and there is some data to suggest this, students who are on the lower end of the scale in terms of
Speaker: of performance were more significantly affected by these disruptions.
Speaker: So their scores decreased by a larger percentage.
Speaker: So it's like these marginal students often in situations where they don't have sufficient resources for their educations or even their lives
Speaker: Their scores fell off a cliff while others were more modest.
Speaker: These are all things that we can dig in deeper and try to figure out over time, not necessarily from the NAEP score, but other tests and other indicators.
Speaker: But we don't know what many people are claiming to know, that in-person school
Speaker: would have made a difference or removing masks would have made a difference or that remote learning cannot work under any circumstance.
Speaker: We do know that students who tend to have less access to good educational resources did even worse than those with better access.
Speaker: But beyond that, we just don't know all that much.
Speaker: We know something really big happened and it's going to take time to recover from it.
Speaker: I used this analogy yesterday in a piece I'm writing and working on.
Speaker: Imagine we're talking about domestic air travel, right?
Speaker: Where over 800,000 people took a flight in 2019 and that dropped to 300,000 in 2020.
Speaker: And it's up over 650,000 or something like that in 2021.
Speaker: If you compare 2021 to 2019, you would say that the
Speaker: Air travel industry is collapsing because we're down from 800,000 to 650,000.
Speaker: The reality is it's in the midst of a recovery after a large shock that has caused significant change.
Speaker: I think that's a much more plausible explanation of what's happening with students.
Speaker: And the idea that these students will be permanently harmed or that they cannot recover from this
Speaker: I think is totally unfounded, counterproductive.
Speaker: And really anybody who wants to say that is pressing an agenda that does not have much to do with trying to understand at a deeper level, at a more practical and actionable level, what can we do to help students in schools today, tomorrow, next year,
Speaker: We're going to continue to experience some level of disruption from this pandemic who are going to continue to have their own issues around burnout, just as their teachers are and are not out of the woods in terms of the effects of this disruption.
Speaker: Certainly.
Speaker: And I think what is so interesting, right, is this just is such a microcosm of
Speaker: what it says about how we use data in education, what data counts, and kind of how we communicate the success of schools and school systems in the general public.
Speaker: A couple of points on this.
Speaker: I think one of my concerns then is just understanding how reactive our political systems are to something like the NAEP score to a New York Times headline.
Speaker: We're going to be making some really
Speaker: dramatic changes to education in response to, in the short term, in response to the singular piece of data that is just going to serve that data.
Speaker: So the conversations that I've seen are around accelerated learning, extending school hours or the school year, even though the United States already has students in schools and American teachers already work more hours and teach, have more contact time with students than their counterparts in the rest of the world.
Speaker: You know, all of these potential solutions.
Speaker: And one thing that I think is particularly interesting, too, is that I don't think it was these results are a surprise.
Speaker: Right.
Speaker: I don't mean to sound callous at all in this kind of sense, too, to your point.
Speaker: We faced a global disruption.
Speaker: We have a million dead Americans.
Speaker: We have hundreds of thousands of orphaned children.
Speaker: And to expect that schools were going to be immune from those changes, we really were just going to be... It was a question of what is the drop going to be in these test scores, right?
Speaker: Because schools were focused on other priorities.
Speaker: Now, the other part of this that I think is interesting and again, not surprising, is that the score drops tracked alongside the groups in society who would have been most heavily impacted by the various waves of the pandemic.
Speaker: You said people in the Northeast, the South, poor communities,
Speaker: black and brown communities that are underserved in so many of the other aspects of public life and access to health care and all of that as well.
Speaker: And I think what is particularly frustrating is our response is not really going to meet the needs of those communities in the sense that we know that it's going to take more resources to sort of, you know,
Speaker: Not just resolve the tensions that are simmering socially, but there's also this leap into the pedagogical side of things, too.
Speaker: So we want to try and solve these systemic social issues of lack of access to health care and poverty, etc.
Speaker: with these pedagogical programs.
Speaker: They're going to try to solve it when they get to the schoolhouse door.
Speaker: Yeah.
Speaker: So kind of to respond to this, I'm curious what your response is here, but what do we say to those people who are going to argue that this data is important, that it's helpful for us in order to address those equity concerns?
Speaker: So it pretty demonstrably shows that people in underserved communities or hit harder than people with more resources.
Speaker: That's not a surprise to anyone who's been in education for longer than five minutes.
Speaker: You know, they would argue we have the data.
Speaker: Here's the data.
Speaker: Here's how it shows how kids who were behind fell behind even further.
Speaker: Again, probably referring to poor kids, black and brown kids, et cetera, that this data is going to help us target those interventions, accelerations, et cetera.
Speaker: Right.
Speaker: There's a lot of sincere people who believe that that's going to be doing what is best for kids.
Speaker: And what what is the response to that?
Speaker: You know, is there is there a response?
Speaker: What's the conversation?
Speaker: Well,
Speaker: I think if there's any clear response to this data, and really this is a response to the NAEP data every single year, it's that under-resourced schools need more resources.
Speaker: And schools by themselves cannot fix larger social ills and lack of access to stable facilities
Speaker: healthy existences.
Speaker: The data says, as we've been talking, if you were in a worse situation in terms of access to high-speed internet,
Speaker: or your own computer, or a quiet place to study, or some adult willing to help, your score dropped more.
Speaker: These are all things that are still in place, even if we go back to school.
Speaker: And school really, you know, this test and schooling in general, is really often a byproduct of the larger societal atmosphere and the things that are going on around students.
Speaker: The notion that
Speaker: this is going to be solved by a kind of accelerated schooling or sometimes what sounds just like a harder cracking of the whip over the heads of students as we try to spur them on to achievement seems misguided to me because as we know, the loss has not only been to their academics, it's been a loss to their overall experience of, of,
Speaker: their humanity, you know, as it has been for all of us, right?
Speaker: Teachers and others aren't burned out only because they've been working under difficult circumstances, either through remote learning or the challenges of trying to teach in person while mitigating the virus.
Speaker: It's that there has been a lot of other stuff going on in our lives around the virus, not the least loved ones getting sick,
Speaker: And potentially dying.
Speaker: So there is a larger recovery to this and a longer recovery to this.
Speaker: One of the points I try to make with people is a year from now, we're going to get next year's NAEP scores and they're going to go up.
Speaker: And people are going to be asking about the innate miracle.
Speaker: How did we return to the previous status quo or close to it in just a year's time when we were 20 years behind last year?
Speaker: And my answer right now, and it'll be the same next year, is time.
Speaker: We need time to process.
Speaker: We need time to quite literally heal.
Speaker: We need time also.
Speaker: And this is the thing.
Speaker: that I wish more of these conversations were about, to figure out what school can and should be going forward.
Speaker: The tremendous hunger to return to the status quo among many corners of the discussion baffles me because there were a lot of criticisms, justified criticisms of the status quo from various parts of the spectrum.
Speaker: from people who I would fundamentally disagree with, they were also critical of status quo.
Speaker: I was critical of status quo.
Speaker: Why are we rushing to go back to something that nobody thought was ideal?
Speaker: Just because it's more comfortable, just because it seems better than what's going on currently?
Speaker: I think those are all natural responses, but I think it's a shame if we don't use this as an opportunity to have
Speaker: better, deeper conversations about what students would benefit from in our schools.
Speaker: So we can use these scores as a way to have those conversations.
Speaker: Unfortunately, that's not what's really been going on, at least not in the first 36 hours since the score's release.
Speaker: Yeah, this is definitely the instant react kind of idea.
Speaker: But I'd
Speaker: really doubt that the discourse is going to get a lot more, a lot healthier after that.
Speaker: It's become this tug of war, right?
Speaker: Of the various forces that want to bend schools to their, to their idea of what schools should be.
Speaker: And so seething on this data point in order to do that was wholly predictable.
Speaker: It's, it's dismaying.
Speaker: It's very difficult to counteract because the first impulse is,
Speaker: is to simply grab your end of the rope and start tugging it in the direction you want.
Speaker: But if we can at least advocate for a smarter conversation, maybe we can make some collective progress.
Speaker: I take the
Speaker: I try to take the good faith of those who see these things differently, seriously, that they also want students to thrive and do well in school.
Speaker: So if that's the case, we should be able to have a better, deeper conversation rather than litigating the past, rather than trying to get in our time machine and decide if students should have been in person sooner or should have been masked or whatever.
Speaker: All that is done.
Speaker: people tried to make the best decision they could have based on what we knew at the time.
Speaker: That's all we can do right now going forward.
Speaker: So let's have a decent look at what's happening so we can make some better decisions.
Speaker: I think that's a great place to end it, John.
Speaker: So thank you so much for spending some time unpacking that with me today.
Speaker: My pleasure.
Speaker: Thanks.
Speaker: Thanks again to John Warner for joining me for that conversation.
Speaker: You can follow him on Twitter at Bibli Oracle.
Speaker: You can also keep the conversation going by following us at HumeResPro and by visiting our website, humanrestorationproject.org.
Speaker: We have an entire Learning Loss Handbook.
Speaker: that is over 50 pages of history and insight into these tests and the learning loss narrative, particularly surrounding the last two years of pandemic schooling.
Speaker: In addition, we have a podcast on the topic of learning loss with Akil Bello and highlights of that same episode available on our YouTube channel if you search for Human Restoration Project.
Speaker: Thanks for listening.
