Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
96: Unpacking Learning Loss: The Who, the What, and the Why
14 Plays3 years ago
Today, Nick and I are going to break down Human Restoration Project's recently released "Learning Loss" Handbook, providing an overview of the ideas we present in the book, the faux narrative being created surrounding "learning loss", and offering an alternative for educators to push for and pursue. You can find the free download at humanrestorationproject.org/materials.
Included in this conversation is...
- The history of the term "learning loss"
- An analysis of standardized testing and what we're measuring
- Showcasing the overall negative impacts of standardized testing
- Deconstructing the financial ties to learning loss and the testing industry
- Dissecting the stereotype threat and Pygmalion effect and its potential connection to the negative narrative of learning loss
Next time we'll break down *how* we counteract this narrative. Stay tuned!
*Apologies for the audio quality, my mic literally broke while recording this podcast! Support our funding drive! :)
FUNDING DRIVE
Human Restoration Project is currently in the midst of our September 2021 funding drive. You can learn more, help us out, obtain donor gifts, and support future podcasts/ resources by visiting Human Restoration Project's website.
RESOURCES
Recommended
Transcript
Introduction & Episode Overview
00:00:06
Speaker
Hello and welcome to episode 96 of our podcast at Human Restoration Project.
00:00:10
Speaker
My name is Chris McNutt and I'm a high school digital media instructor from Ohio.
00:00:14
Speaker
Before we get started, I want to let you know that this was brought to you by our supporters, three of whom are Kate Rowbottom, Kimberly Baker and Rachel Lawrence.
00:00:22
Speaker
Thank you for your ongoing support.
00:00:23
Speaker
You can learn more about the Human Restoration Project on our website, humanrestorationproject.org or find us on Twitter, Instagram or Facebook.
00:00:45
Speaker
Today, I am joined by Nick and we are going to break down Human Restoration Project's recently released Learning Loss Handbook, providing an overview of the ideas that we present in the book, the faux narrative being created surrounding learning loss, and offering an alternative for educators to push for and pursue.
The Learning Loss Handbook's Purpose
00:01:01
Speaker
You can find a free download at humanrestorationproject.org slash materials.
00:01:06
Speaker
So Nick, we are here and we are talking about the handbook.
00:01:11
Speaker
And we were just saying, hey, let's just go through the entire thing.
00:01:15
Speaker
Talk about what's in here, what we learned, the struggles along the way of trying to comprehend this.
00:01:22
Speaker
And basically what we're looking at here.
00:01:24
Speaker
And we had said before that basically learning loss handbook, in our view,
00:01:28
Speaker
is the best resource as of right now to understand what's going on.
00:01:32
Speaker
Because what we did is we tried to combine a bunch of folks that were, I guess, way smarter than us, and all assemble them into one place so that we can really tease this apart.
00:01:44
Speaker
That was like the genesis of this idea is that throughout the pandemic, going back as far as panic, like the moral panic over schools in the spring of 2020 when COVID was just getting started, I don't recall necessarily the learning loss narrative
Origins of the Learning Loss Narrative
00:02:00
Speaker
there.
00:02:00
Speaker
Maybe it started to pick up over that summer 2020 and into the fall as schools mulled over hybrid online remote options, but
00:02:08
Speaker
you know, by the spring of 2021, it really had kind of solidified into that solid narrative of like something profound is being lost here, but it didn't seem to be focused on what I think we felt were the right things that had been lost in those now 19, 20 months of the pandemic.
00:02:31
Speaker
So we
00:02:32
Speaker
we didn't see anything out there that had addressed it, I think, from that perspective.
00:02:36
Speaker
But I was just going back through and kind of seeing some of the things that led us to release it when we did, because it was it had been a project that had been in place for months.
00:02:47
Speaker
And it was really curious that that one of the things that kind of caused us to, you know, put in the last, the last kind of pull an all nighter and get that thing out there before the end of August was,
00:02:59
Speaker
An article that came out, an op ed in the New York Times, the school kids are not all right.
00:03:06
Speaker
And the whole piece was focused on this data that had been extrapolated out from from the learning loss narrative, not just to suggest that there would be academic experiences.
00:03:18
Speaker
academic gaps or academic learning loss that would need to be recovered, but actually that this could be extrapolated into the loss of future potential incomes.
00:03:30
Speaker
Even as far as to say the impact on the US economy could range from 128 billion to 188 billion every year as the cohort enters the workforce.
Profit Motives Behind Learning Loss
00:03:40
Speaker
So just this, this huge moral panic
00:03:44
Speaker
not around the sort of the socio-emotional impact on kids, not the impact on the lives that were lost and kids that we know were orphaned and are now in worse life situations because of the ongoing impact of COVID, but this asinine idea that because
00:04:06
Speaker
kids had an adapted school experience or maybe didn't finish their 2020 school year as they would normally have done, that that is going to lead to a drop in lifetime earnings and US GDP felt like that was the wrong thing to do.
00:04:27
Speaker
So for us, I think it was trying to refocus on what matters and what had mattered during that time.
00:04:35
Speaker
And then try to try to dispel a lot of those myths that had popped up around learning loss, where that data was coming from, and try to give some context to that, because it kind of seemed like the
00:04:51
Speaker
the media portion of it was sort of meant to circumvent that frontal cortex where you think about things and just go right to the amygdala and just turn on that fight or flight response and demand, you know, unsafe conditions for school, which I think we're seeing now has backfired in a big way.
00:05:10
Speaker
Before we move on and talk about more about what we have in this handbook, I do want to reiterate the fact that we're not claiming that learning loss didn't occur in terms of a major disaster didn't occur in our lives.
00:05:24
Speaker
I guess there's no issue going on that something wasn't lost.
00:05:27
Speaker
What we are claiming is that the narrative surrounding learning loss, as in the one that's been propped up by the McKinsey report, which we'll talk about in a second, as well as the testing companies in general, serves to earn profits for testing companies.
Handbook Structure & Educational Approaches
00:05:44
Speaker
It serves to have districts and therefore students
00:05:48
Speaker
take a bunch of additional tests, purchase a bunch of test remediation software, and do many things to increase their learning loss, quote unquote, from the pandemic.
00:06:00
Speaker
That is not the same as reestablishing connections.
00:06:04
Speaker
I guess the TLDR of the handbook is stop focusing on purchasing products that will increase test scores because that will lead to a lot more problems than it might attempt to solve.
00:06:15
Speaker
And instead focus on connections, rebuilding community, and social emotional learning.
00:06:20
Speaker
I mean, it was a global pandemic.
00:06:22
Speaker
It's still going on.
00:06:23
Speaker
And it's a major problem.
00:06:25
Speaker
The handbook itself is divided into two parts.
00:06:27
Speaker
The first part, which we'll start with is, okay, what is learning loss?
00:06:31
Speaker
Where does that come from?
00:06:32
Speaker
How is it being used?
00:06:33
Speaker
How are we making these claims?
00:06:34
Speaker
What's its connection to the standardized testing industry?
00:06:36
Speaker
And part two is, okay, knowing all that, what can we do instead?
00:06:41
Speaker
What other connections is this?
00:06:43
Speaker
How can we reimagine
Historical Context of Education Narratives
00:06:44
Speaker
systems?
00:06:44
Speaker
How can we build back better?
00:06:47
Speaker
And I think that really the narrative is sadly timeless or it's cyclical.
00:06:54
Speaker
This is something that's not new.
00:06:56
Speaker
Even though we're talking about learning loss in a pandemic context, so 2019-2020 school year, 2020-21 school year, hopefully not longer than that, it's still going on with summer slide.
00:07:08
Speaker
the quote unquote learning crisis, even things like nation at risk and previous reports where test scores have been hyper low.
00:07:15
Speaker
And as a result, testing companies have said, hey, we need to buy all this stuff.
00:07:19
Speaker
We need to fix this.
00:07:20
Speaker
Have your kids do all these things.
00:07:21
Speaker
And I'm sure most teachers listening in have been told at one point or other to use some kind of testing software or test prep companies materials to
00:07:31
Speaker
had a kid sit in front of a computer or do some worksheet and say, hey, we're going to increase test scores.
00:07:35
Speaker
And usually it's absolutely miserable.
00:07:37
Speaker
Our claim is it's not really rooted in anything that's relevant.
Critique of Standardized Testing
00:07:42
Speaker
And hopefully we prove that here today.
00:07:45
Speaker
We're just going to start with it.
00:07:47
Speaker
Nick and I were just talking about how this is kind of like the audio narrative over the DVD special edition.
00:07:53
Speaker
We're going to talk about everything about what happened in this Learning Laws Handbook, where it came from, and hopefully give you all a good idea about what it is that we're trying to talk about.
00:08:03
Speaker
So the very first thing that we started talking about was just terminology and history.
00:08:09
Speaker
The beginning is just breaking down whatever the heck is going on when it comes to this.
00:08:15
Speaker
And I still remember in our first meeting when we were just trying to dissect the reporting around learning loss, it took a solid hour or two just to understand what exactly...
00:08:29
Speaker
This is it's very confusing.
00:08:33
Speaker
And I wouldn't be surprised if there is at least one error in this learning loss handbook, because a lot of the stuff doesn't make sense.
00:08:40
Speaker
Perhaps it doesn't make sense by design.
00:08:43
Speaker
Like, like, it's kind of like just a bunch of people working around something that just doesn't work.
00:08:47
Speaker
So therefore, it just doesn't make sense.
00:08:49
Speaker
Or maybe it's just, I don't know, I don't really know how else to put it.
00:08:53
Speaker
That's the thing I think that is really interesting, too, in the way that we have continued to kind of seed a lot more power into those groups that claim to be generating the data that we would make our data-driven decision-making from.
00:09:10
Speaker
Yeah.
00:09:11
Speaker
But a lot of times people in education who work in the classroom or are closer to the classroom, right, aren't really equipped with the toolkit to understand what that data is saying.
00:09:24
Speaker
So as we as classroom teachers began to dig into that data, it came with a whole new terminology, a whole new jargon that we really had to dig into to ask, what
00:09:36
Speaker
not just what this is saying, but then is this thing significant?
00:09:39
Speaker
So we had to understand the language of the standardized tests in order to be able to interpret whether a one to 3% difference is significant or if that's even important, right?
00:09:55
Speaker
Are these things worth spending millions and millions and millions of dollars to try and recover?
McKinsey's Influence on Learning Loss Narrative
00:10:01
Speaker
Like, what is this even measuring?
00:10:03
Speaker
So yeah, so that's why we start this thing off with, with basically a glossary, because a lot of the terminology is fly by night, like it's new, like what is accelerated learning versus learning loss or learning recovery?
00:10:16
Speaker
Are those just synonymous for things?
00:10:18
Speaker
There's been a lot of internet debate about what the appropriate use is of those terms, but they're the tip of the iceberg of that same kind of idea that goes back not just to the summer slide and those notions that in the seat time away from school, something is lost if we're not constantly cramming or pouring things into kids' brains.
00:10:43
Speaker
It's just going to ooze out of one side or the other, but really then
00:10:48
Speaker
I don't know if it's because we're addicted to context, because like we're, we were, we were social studies teachers, you know, or something, but, but we felt like we had to go all the way back to like what the early 1900s and talk about like, basically, the 20th century, the industrial revolution in the 20th century and the advent of
00:11:10
Speaker
of management and this concept of Taylorism and basically trace that Taylorist idea, you know, that was used to fine tune factories back in the day, clear up through what, clear up through the Cold War and Sputnik.
00:11:27
Speaker
And then, of course, in the 1980s through a nation at risk.
00:11:32
Speaker
And then up through then, of course, No Child Left Behind,
00:11:37
Speaker
In the in the early 2000s, a bipartisan, you know, push to a push towards standardization and the punitive use of assessments and the new accountability movement.
00:11:49
Speaker
And then, of course, President Obama's own version of that with the race to the top, kind of putting, you know, putting the.
00:11:57
Speaker
financial incentives in place for schools and districts and states and stuff who could adopt these test score delivering programs clear through, gosh, clear through today, right, with McKinsey and company.
00:12:11
Speaker
And I think they are probably the most cited in this whole learning loss narrative.
Alternative Student Assessment Methods
00:12:18
Speaker
So then, yeah, a little bit later on, we go a little bit more into McKinsey.
00:12:23
Speaker
But the whole premise of this thing is, like, measured on this Taylorist notion of seat time as sort of a method of inputting information.
00:12:35
Speaker
And the lack of that seat time means that there must be a loss of that information.
00:12:40
Speaker
And being able then to what be able to break these things down into, well, X number of days missed means you are this many units of learning, you know, behind or what you have lost from where you were previously.
00:12:54
Speaker
So we trace it back to Taylorism, but also the fact that.
00:12:58
Speaker
McKinsey, let's talk about McKinsey right now.
00:13:01
Speaker
McKinsey is a global consulting firm, like they are rooted in foreign policy, but specifically, their interests lie in like restructuring, usually management firms or companies, they're the folks that go into companies and will fire a bunch of people in order to try to increase their stock prices, kind of like old, I guess, Mitt Romney esque
00:13:27
Speaker
or I guess more modern day Pete Pudiech worked for McKinsey.
00:13:31
Speaker
McKinsey has been tied to the opioid crisis.
00:13:35
Speaker
They've been tied to some shady arms deals and work in Afghanistan.
00:13:40
Speaker
They have a pretty seedy history in general.
00:13:44
Speaker
And they've also worked at just basically spreading the idea of different business or neoliberal style practices to every facet of America and where America is, aka
00:13:57
Speaker
the world this this connection of learning loss is based off of the idea of global competitiveness through testing whenever we do testing in the united states we center our work on are we keeping up with japan or with france or whoever with our math and reading scores and
00:14:18
Speaker
If our math and reading scores are not, McKinsey, as well as the folks who draw upon McKinsey's work, believe that by using Taylor's principles or by just standardizing this model and putting more inputs in and treating school like a corporation, that you will see increased results.
00:14:39
Speaker
But sadly, as you and I both know, and I think most people listening, we're
00:14:42
Speaker
School is not a corporation.
Standardized Testing & Systemic Bias
00:14:44
Speaker
It doesn't work like that.
00:14:45
Speaker
You do not learn through neo-economic ways.
00:14:50
Speaker
You can't just hire a new manager and then just say, okay, everything's better now with all of the different problems that face the American school system.
00:14:58
Speaker
And it's not the teacher's fault per se.
00:15:01
Speaker
It's all of the underlying systems, including things like funding, solving issues of inequity in society, et cetera.
00:15:08
Speaker
So fast forward over about 50 years since McKinsey was founded, you have, as you just said, No Child Left Behind, Race to the Top, Nation at Risk.
00:15:18
Speaker
These are all just different reports that say American kids are behind, there's a giant achievement gap, et cetera, et cetera.
00:15:24
Speaker
So put more money into schools to address standardized testing so we can figure out what's going on and try to make things better.
00:15:31
Speaker
I think that that's rooted in perhaps good ideas.
00:15:36
Speaker
I don't think that everyone that believes in those things is necessarily doing it for nefarious purposes, but there's certainly a very strong lobbying wing that invests in this.
00:15:45
Speaker
That Taylorism, right, is rooted in the efficiency movement.
00:15:49
Speaker
You know, and that notion that, you know, what could work to either induce workers to work faster or churn out more widgets or however we could speed up the assembly line kind of practices there.
00:16:05
Speaker
That's what that kind of mechanical...
00:16:07
Speaker
management efficiency movement is rooted in.
00:16:10
Speaker
And it's just that mindset, but applied to schools, kind of like as a conveyor belt.
00:16:16
Speaker
And so, right, the notion that someone is at a third grade level or at a sixth grade level or at a whatever level kids are supposed to be at is, I mean, it's completely arbitrary, right?
00:16:27
Speaker
And determined by whom, you know, by
00:16:30
Speaker
the makers of this particular test by, is it determined by community standards, right?
00:16:35
Speaker
Is it determined by national standards?
00:16:37
Speaker
Is it by global standards?
00:16:38
Speaker
By whose standards are we judging where these kids are at?
00:16:41
Speaker
So, you know, I imagine my seniors now that I have, who have had, right, two, three years now, gosh, it seems like forever, but, you know, they're the tail end of their sophomore year and then their entire junior year and now heading into their senior year disrupted, you know,
00:17:00
Speaker
by the pandemic, right?
00:17:03
Speaker
It doesn't fit into that efficiency conveyor belt mentality because if this pandemic has shown anything, it's that human beings and human life is messy and inefficient, particularly in times of crisis.
Shift from Test Requirements in Higher Education
00:17:16
Speaker
So why would education be anything but messy and inefficient in this time as well, right?
00:17:22
Speaker
The notion that we would have stayed solid and steady on this conveyor belt or should have stayed solid and steady on this conveyor belt
00:17:29
Speaker
is something that I think the folks pushing this learning loss narrative have never addressed.
00:17:35
Speaker
What would be the acceptable level of disruption given the malevolent mishandling and mismanagement of this pandemic in the United States of America?
00:17:47
Speaker
We can point to international examples and say that, oh, kids in such and such country are going back to schools as quote unquote normal and pushing for those same ideas here when those other countries have just done, they've done the other public health work to make those things possible and just we haven't.
00:18:06
Speaker
So there's a lot else tied up in this notion, but the idea that we can distill that
00:18:33
Speaker
That's rebuilding the future on the college board's values.
00:18:36
Speaker
That's right.
00:18:37
Speaker
And Pearson's values and the people, again, who are giving us the data that we're using for data driven decision making and then turning around and buying their products.
00:18:48
Speaker
Right.
00:18:49
Speaker
to resolve this crisis.
00:18:51
Speaker
So, you know, it's going to sound conspiratorial, but it's really, if you just peel the thing away, it's just about, you know, power and money is really what it is.
00:19:00
Speaker
And it's about, you know, them providing money
00:19:05
Speaker
marketing materials to solve a problem that they have the assessment data in their minds to resolve.
00:19:12
Speaker
And so it's a financial incentive for them to draw this into a hysteria so that way they can get schools to buy their accelerated computer software or to fund their summer academies.
00:19:26
Speaker
I have a feeling that we're probably going to spend the most amount of time just talking about that.
00:19:30
Speaker
But I do want to make sure that we start off with square one or step one, which is just, does any of it matter to begin with?
00:19:38
Speaker
So if we pull back the curtain and we just say, let's imagine that these reports never came out, does the standardized test actually measure anything of use?
00:19:48
Speaker
Like if we just look at the questions and we look at what standardized tests do and we look at the ramifications of issuing them,
00:19:56
Speaker
Does it even matter if hypothetically there were points that were lost or students are falling behind?
00:20:02
Speaker
Like, what exactly are we measuring?
Problems with Standardized Test Questions
00:20:05
Speaker
And we started this by simply looking at a series of different questions from a variety of different tests we pulled from.
00:20:13
Speaker
The MAP, the NAEP, the STAR, a couple of state tests, the PISA, all the acronyms.
00:20:19
Speaker
Yeah, all the different acronyms, all the different tests that kids have to take.
00:20:24
Speaker
Freshman year, our students last year, I believe, had to complete 17 different tests in the last two weeks of school.
00:20:32
Speaker
So it's just brutal.
00:20:33
Speaker
It is brutal how many tests pop up.
00:20:36
Speaker
Yeah.
00:20:37
Speaker
And we didn't, I was the one that threw these questions in this book, and we'll read a few of them here in a second.
00:20:43
Speaker
I didn't cherry pick these.
00:20:44
Speaker
Like I didn't Google like hardest test questions or like search through and browse a bunch of test questions before I chose one.
00:20:51
Speaker
It was usually like one of the first three.
00:20:53
Speaker
And I kind of chose them at random just because they're all absurd.
00:20:57
Speaker
We'll start with the ones that are supposed to be easy here.
00:21:00
Speaker
We'll do a few.
00:21:01
Speaker
So there's a test question on here from the map.
00:21:04
Speaker
It's an adaptive test.
00:21:07
Speaker
So you could get this question at any point in elementary school, depending on your quote unquote reading level.
00:21:11
Speaker
And it's two paragraphs about swimming.
00:21:15
Speaker
And it's like Alex says, swimming is really fun.
00:21:17
Speaker
It's really easy to learn how to swim.
00:21:19
Speaker
My favorite place to swim is the lake near my home.
00:21:20
Speaker
I do not like swimming in pools.
00:21:21
Speaker
Pools are too crowded.
00:21:23
Speaker
And then Carl says, swimming is hard work.
00:21:25
Speaker
It took me all summer to learn how to swim, but I'm glad I learned.
00:21:27
Speaker
I like going to the pool to swim.
00:21:29
Speaker
I have fun swimming with my friends.
00:21:30
Speaker
And then it has you sorting between all the things that Carl and Alex agree on and things that they disagree on.
00:21:39
Speaker
You have to like pull them into the right sections.
00:21:42
Speaker
And what we were curious about is if you are reading this prompt, first off, could you answer it?
00:21:47
Speaker
I think most adults probably could answer the question.
00:21:50
Speaker
But I think the better question for this one is like,
00:21:54
Speaker
Why might a student answer this question correctly or incorrectly?
00:21:58
Speaker
What barriers could fall in there?
00:22:01
Speaker
This is on page 13, if you're looking in the book at the same time that we are.
00:22:05
Speaker
But I think the thing that happens when I see this is that there's just cognitively so much going on on the page for this question.
Impact of Testing on Teaching Practices
00:22:15
Speaker
Like if I'm a child and taking this test, it's just really hard to figure out the formatting and then to figure out what the question is asking and then processing, how does it want you to answer it?
00:22:29
Speaker
So something that could be very easily resolved, you know, or assessed between students and teachers just by having, making this a conversation, you know, and then you can pretty easily sort out what kids, you know, get and what they don't get, turns it into, I don't know, kind of like a cognitive nightmare, honestly, just trying to sort through this piece.
00:22:50
Speaker
And yeah, clicking and dragging and moving around.
00:22:52
Speaker
So there's like the technological elements of this thing too.
00:22:56
Speaker
Yeah, it's just...
00:22:58
Speaker
It's just kind of a nightmare question, honestly.
00:23:02
Speaker
Well, also, there's a cultural component to this.
00:23:05
Speaker
This has been heavily critiqued.
00:23:07
Speaker
The narrative surrounding this typically brings up cultural critique and the fact that there is a sizable population of people that don't swim in pools, that don't have a pool near them.
00:23:20
Speaker
If this is aimed at elementary school students, they might not have any idea...
00:23:25
Speaker
what this prompt is really even talking about.
00:23:27
Speaker
They might hypothetically know what these things are, but it's really hard, especially when you're in first or second grade, to visualize and answer a question like this without the experience of knowing what these things are.
00:23:42
Speaker
I could see students struggling with this, but to me, the bigger part is that's just really boring.
00:23:48
Speaker
If I've read 10 of these prompts, I don't care anymore.
00:23:52
Speaker
Who cares?
00:23:52
Speaker
cares if swimming is really fun and what Alex thinks.
00:23:56
Speaker
What bothers me about this is that is the data useful to know?
00:24:01
Speaker
Yeah, of course it's useful to know if like a kid can't read and they're in elementary school.
00:24:05
Speaker
Like we want to make sure kids can read.
00:24:07
Speaker
But this is not motivating kids to want to read to begin with.
00:24:13
Speaker
Like you could easily issue a test like this between teacher and student, if you're a support specialist or something, where you can figure out this exact same data.
00:24:22
Speaker
But you could do it in a way where the kid actually enjoys doing the thing that they're being measured on and you'd be helping their way through it.
00:24:28
Speaker
Right.
00:24:29
Speaker
This is this is not something, again, that could not just be assessed in the moment by a teacher and a student that would take an inordinate amount of time to sort out.
Detrimental Effects of Testing on Education
00:24:39
Speaker
Right.
00:24:40
Speaker
take a day to look at the data if you need to, and then be actionable like the next day.
00:24:44
Speaker
I don't know what why this has to be on a map question.
00:24:46
Speaker
Now, when you brought up that cultural competency piece, it made me think of some questions that didn't actually make it in here, because I think we had far more pages than ended up being in here.
00:24:57
Speaker
And one was actually about a toll booth.
00:24:59
Speaker
Do you remember this question?
00:25:01
Speaker
It's on page 15.
00:25:03
Speaker
Oh, is it?
00:25:04
Speaker
Oh, okay.
00:25:05
Speaker
Well, that's great.
00:25:05
Speaker
There it is.
00:25:06
Speaker
So that's the same kind of idea, right?
00:25:08
Speaker
The toll rates for crossing a bridge are 650 for a car and 10 for a truck.
00:25:12
Speaker
Now, you know, in Iowa, where we don't have a toll for anything, right, you might get to 10th grade and not have an understanding of what toll roads are, if that's not something that you've experienced or something that you know about.
00:25:25
Speaker
So, so then in that question, it's not even asking you, it's
00:25:31
Speaker
it's a it's asking you like to do hardcore algebra with this.
00:25:35
Speaker
So so then which of the following systems of equations yields the number of cars and the number of trucks that cross the bridge during the two hours?
00:25:45
Speaker
So you have to turn this concept that you don't understand into this mathematical equation.
00:25:49
Speaker
But meanwhile, you're wondering, what the hell is a toll rate?
00:25:52
Speaker
What's a toll bridge?
00:25:54
Speaker
I've never driven a truck across that.
00:25:55
Speaker
Why?
00:25:56
Speaker
Why is that there?
00:25:57
Speaker
So that question.
00:25:58
Speaker
So that question is from the SAT.
00:26:01
Speaker
for someone that can't see it, it's asking you to input these values into x and y to make the equation that you would use to solve this.
00:26:10
Speaker
So for example, x plus y equals 187, 6.5x plus 10y equals 1,338 divided by 2.
00:26:13
Speaker
And it's one of those things where I do not know a person alive
00:26:23
Speaker
who would ever solve this question like this like this might be mentally what you're doing like with a calculator in your head but no one would write this out as x plus y and put it into some it feels like it's forcing this to make up for the fact that you had to learn this very specific standard to meet this test it's not rooted anything realistic and again just like
00:26:45
Speaker
If this is my 10th test in two weeks, and not to mention, this is the SAT, so I'm super anxious about it, who cares?
00:26:55
Speaker
Who in their right mind would do it?
00:26:56
Speaker
Who cares how many trucks and cars pass a toll road?
00:26:59
Speaker
Even if you know what toll roads were,
00:27:01
Speaker
All I care about is, do I have enough money to pay for the toll?
Disconnect between Tests and Educational Goals
00:27:05
Speaker
I don't know.
00:27:05
Speaker
It's a very silly question.
00:27:08
Speaker
What I wanted to bring up is the one right before that on page 14 from the star.
00:27:12
Speaker
I think this is the fan favorite for us.
00:27:14
Speaker
I still can't pronounce this word.
00:27:16
Speaker
I still can't.
00:27:16
Speaker
I'm going to embarrass myself because I don't remember either to bring it up.
00:27:19
Speaker
And we've looked it up several times.
00:27:21
Speaker
The star is another one of the plethora of tests that's used to gauge student understanding for multiple times during the year, kind of like the map test.
00:27:31
Speaker
from Renaissance Learning.
00:27:32
Speaker
So this is a 12th grade test.
00:27:33
Speaker
So this is supposed to be the hardest test that they issue.
00:27:37
Speaker
It's from English.
00:27:38
Speaker
Nick, do you know how to pronounce it?
00:27:40
Speaker
Did you look it up?
00:27:41
Speaker
I do because I just Googled it.
00:27:42
Speaker
So how is the word actually pronounced?
00:27:44
Speaker
Synecdoche.
00:27:45
Speaker
Okay, synecdoche, which to be fair, Thomas, who is an English teacher on our staff, knew what this word was.
00:27:53
Speaker
Kind of.
00:27:54
Speaker
However, it took all three of us studying this question to figure out what the right answer is.
00:27:58
Speaker
So quiz for you at home, see if anyone knows this.
00:28:01
Speaker
It says synecdoche is a figure of speech in which a part of something is used to mean the whole of, or conversely, the whole for the part.
00:28:10
Speaker
One example would be if someone referred to her car as her wheels, which is another example of synecdoche.
00:28:18
Speaker
One, the captain called all hands on deck.
00:28:22
Speaker
Two, the pen is mightier than the sword.
00:28:24
Speaker
Or three, the dawn raised her rosy red fingers.
00:28:28
Speaker
I feel like I'm on a game show.
00:28:30
Speaker
It legitimately sounds... What's wild to me is this one gives you an example of what it is, which in theory is supposed to supply context.
00:28:37
Speaker
But I'm not gonna, I know we discussed this before.
00:28:39
Speaker
I still do not know what the right answer is.
00:28:41
Speaker
I do not know which of those three is the right answer.
00:28:44
Speaker
And there's only three.
00:28:45
Speaker
That's the thing.
00:28:46
Speaker
It's not even like the traditional four, like on some of them.
00:28:49
Speaker
Yeah, so like the first hurdle obviously is, can you even pronounce this word correctly?
00:28:53
Speaker
And if you can't,
00:28:55
Speaker
Is it that important knowing anyway?
00:28:57
Speaker
I mean, if you were pronouncing this, you would literally read synekdouche.
00:29:03
Speaker
That's just like how you would phonetically pronounce that.
00:29:06
Speaker
Or synekdouche.
00:29:06
Speaker
Synekdouche.
00:29:08
Speaker
Yeah, we're just embarrassing ourselves now, but
00:29:12
Speaker
We have college degrees.
00:29:13
Speaker
We are college educated, successful, right?
00:29:16
Speaker
Like successful in big scare quotes, right?
00:29:18
Speaker
But like functional maybe is the correct word here, right?
00:29:22
Speaker
Like have functional adult lives.
00:29:25
Speaker
Like I've got children, you know, like we both have homes and cars and like, you know, we manage these things in our lives.
00:29:33
Speaker
And like at no point in my life has there been a gate that has been kept closed to me because I have not understood synecdoche.
00:29:40
Speaker
So I think really that's kind of the lens then that we were thinking about with these questions.
00:29:47
Speaker
Somewhat of the absurdity of them aside, I mean, you can kind of quiz yourself at home or do it yourself if you'd like to.
00:29:53
Speaker
But we were really asking, yeah, what is this really assessing?
00:29:57
Speaker
For students who might be unmotivated or disinterested, what would cause them just to...
00:30:03
Speaker
You know, if they couldn't 50 50 and guess just look at the prompt feel overwhelmed fill in a bubble and move on and then how accurate then is the information that we're getting is that actionable in the first place because if we're not acting on whether or not kids know we're acting on whether or not they care to answer the question in the first place so you know we're responding to the wrong thing.
00:30:23
Speaker
Right.
00:30:24
Speaker
Also, if we were acting on if kids knew, then we wouldn't receive almost the vast majority of these test scores partway through the following year.
00:30:32
Speaker
Almost all of these tests, you don't get the data anyway.
00:30:35
Speaker
And even when you do, so much time has passed and that usually teachers have changed that there's no way for you to even act on the data.
00:30:43
Speaker
The data just serves to showcase typically the problem more than it does to actually offer any kind of solution for you outside of purchasing more test prep materials.
00:30:52
Speaker
And before maybe we wrap up this section, I'll kind of maybe with an anecdote here for me.
00:30:57
Speaker
Really quick.
00:30:57
Speaker
I'm pretty confident, by the way, the answer is three.
00:31:00
Speaker
Dawn raised her rosy red fingers because the trivia side of me wants to have that answer right.
00:31:04
Speaker
But I'm not positive.
00:31:05
Speaker
That's okay.
00:31:06
Speaker
So somebody at home listening right now, send us an email.
00:31:10
Speaker
Chris at human restoration project.org.
00:31:12
Speaker
Don't send it to me.
00:31:13
Speaker
Send it to Chris.
Cultural Bias in Testing
00:31:15
Speaker
But in my own experience, I remember, I don't know, I had been in my district maybe three years, maybe four years.
00:31:20
Speaker
So I was fairly new to the district and at that point fairly new to the profession as well.
00:31:27
Speaker
And I can just remember us being called into a meeting where we were meeting with somebody from the Iowa assessment team.
00:31:34
Speaker
Like they were part of the test development team or part of the research team, something like that.
00:31:39
Speaker
And they were going to come and talk to us about the social studies portion of the like then the Iowa assessment in its form, maybe back in the mid 20 teens or something like that.
00:31:48
Speaker
Right.
00:31:49
Speaker
And I can just remember sitting in that meeting and kind of just being excited at first because it was like, oh, we're going to pull back the curtain and kind of see, you know, how the sausage gets made on this kind of stuff.
00:31:59
Speaker
And maybe that'll help us with the responses.
00:32:01
Speaker
I mean, our scores weren't bad to begin with anyway, but the more that we went into the actual test, which as educators was the first time that I had actually gotten to see the test, they handed us the test booklet.
00:32:12
Speaker
I still have it.
00:32:13
Speaker
They handed us the test booklet and we went through the whole exam.
00:32:17
Speaker
So this is what students take.
00:32:20
Speaker
What's your response?
00:32:21
Speaker
How can we improve our teaching in response to what we know now?
00:32:25
Speaker
And the answer was, we're not going to change a damn thing because what we're doing is head and shoulders above anything that this test expects our kids to do.
00:32:34
Speaker
We're getting kids to, uh, you know, write, uh, write letters to, to local elected officials, right.
00:32:40
Speaker
And, and kind of learn about that process.
00:32:43
Speaker
We're having them go to school board meetings.
00:32:45
Speaker
We're right.
00:32:46
Speaker
Having them do, um, experiential kind of project-based learning stuff.
00:32:50
Speaker
We're getting them to think and read, uh, like a historian, you know, we're getting them to, um,
00:32:56
Speaker
You know, understand the impact of economic thinking on everyday decision making and all these other kinds of things.
00:33:03
Speaker
And the test wanted to try and reduce that whole experience into a series of the most inane, the most irrelevant questions that I could ever imagine.
00:33:15
Speaker
So I left the meeting just being disgusted.
00:33:18
Speaker
And the decision that we made as a department was, well, we're not going to do anything in response to this because it's not worth responding to.
00:33:26
Speaker
And we never have.
00:33:27
Speaker
We've never been responsive to these tests.
00:33:29
Speaker
And we're not like a big tested subject.
00:33:31
Speaker
So it hasn't mattered.
00:33:32
Speaker
But that's at the end of the day.
00:33:34
Speaker
It's like, is what you're doing in your classroom right?
00:33:37
Speaker
right by students or in front of you?
00:33:40
Speaker
Or are we trying to impress, right, the people who are furthest from students with the researchers for your state assessment and to try and, you know, to try and juke the stats a little bit to show progress in those areas?
00:33:56
Speaker
Or are you teaching the kids
Influence of Educational Narratives on Policies
00:33:57
Speaker
in front of you?
00:33:57
Speaker
There's so much cognitive dissonance here because what a test manufacturer and district will tell you is that in theory, you are not supposed to be preparing students for the test.
00:34:09
Speaker
These tests are meant to just see whether or not teaching is effective and you're not supposed to cater your content towards the test.
00:34:16
Speaker
You're supposed to cater your content towards the standards, which are then addressed by the test.
00:34:20
Speaker
However, at the exact same time, every district does this.
00:34:25
Speaker
Every district cares about their test scores because typically your funding is associated with your test scores.
00:34:30
Speaker
And every test company, either themselves or through some kind of partner, offers resources that do increase traditionally test scores.
00:34:39
Speaker
And the way that they do that is by teaching students exactly how to answer the questions, not because they necessarily understand the content, but because they know these very specific clues and hints and types of questions that this content asks.
00:34:54
Speaker
And the same reason why I'm sure everyone had a friend in high school who was great at taking tests, but ultimately had no idea what was going on.
00:35:04
Speaker
Because the tests only measure a very, very, very specific subset of knowledge and don't accurately measure learners at all.
00:35:12
Speaker
I myself am a terrible test taker, but I think I can produce things and do things quite well.
00:35:18
Speaker
And also we highlight on here, I don't want to get too caught up on it, but...
00:35:22
Speaker
We highlight on here, I mean, there's a lot of data about standardized testing and the problems it cause.
00:35:28
Speaker
There's anxiety, sleep deprivation, fear, assume cultural knowledge.
00:35:34
Speaker
It attracts from creativity.
00:35:36
Speaker
I mean, modern day tests that are created by AI in order to speed up the process, not only are the multiple choice test questions, okay, whatever.
00:35:43
Speaker
I mean, obviously they are quickly graded, but now they do writing through AI.
00:35:48
Speaker
And there have been multiple circumstances where folks have just written nonsense.
00:35:52
Speaker
And the test is like, okay, that's a five out of five because you use this certain amount of words and these certain sentences.
00:35:57
Speaker
No one actually cares what you wrote.
00:35:59
Speaker
So if you have an original idea or you're a creative writer, who cares?
00:36:02
Speaker
Did you use the word synecdoche in your paper?
00:36:04
Speaker
In which case that's a five out of five.
00:36:06
Speaker
And just the whole legal apparatus that surrounds this, even pulling these questions and putting them on this document,
00:36:15
Speaker
I had to use previous versions or things from test prep companies.
00:36:19
Speaker
I can't actually pull up the test myself and take screenshots of it because that would be illegal.
00:36:24
Speaker
Test companies own the rights and don't allow folks to share the data.
00:36:28
Speaker
I mean, I'm sure everyone has probably given or been around, like giving like the SAT, ACT, etc.
00:36:33
Speaker
or you have to read the ridiculous prompt that's like if you share this date on on social media we're going to come after you and you could be like sued like what like your test results will be deemed inaccurate it's like why are we making tests that we can't prepare for by looking at the test first that meant to me makes no sense
00:36:50
Speaker
The irony of leaving that meeting, you saying teach the test prompted this.
00:36:56
Speaker
The irony of that meeting, and this has kind of become a running joke in our department for some time, is we were advised by the IO assessment person to teach, or what was their line?
00:37:08
Speaker
Yes, teach to the test, because that's going to get you the results that you want.
00:37:13
Speaker
Teach to the stuff that's on the test.
00:37:15
Speaker
You just can't teach the test.
00:37:19
Speaker
So we're like, isn't that just more difficult?
00:37:22
Speaker
Like if I know what's going to be on the test, why not just give them the same kinds of questions and things that are going to be on there?
00:37:28
Speaker
Like if that's what matters, why wouldn't you just spend all of your time doing the things that are going to improve those test scores instead of, you know, to your bigger point, you know, if you have some standards that you can draw from a broad range of skills, abilities, content, time periods, like
00:37:45
Speaker
thinking from a social studies background, why would you not do that?
00:37:49
Speaker
But those are going to be things that may or may not reap rewards for you on the test later on.
00:37:55
Speaker
I mean, I teach an AP class.
00:37:56
Speaker
This is the world that I live in is, you know, this constant tension between, you know, giving my students what I think they deserve out of that classroom experience and teaching to a single three and a half hour exam in May.
00:38:12
Speaker
And that's the tension that we're at at a systemic level.
00:38:17
Speaker
K through 12 is teachers.
00:38:19
Speaker
Are you teaching kids to 360 days of the year?
00:38:24
Speaker
Or are you teaching to those five days that they're going to be trapped in MAP tests or whatever the case might be?
00:38:30
Speaker
Yeah.
00:38:31
Speaker
And also just the fact that the way that you fix this isn't by attempting to improve the test.
00:38:36
Speaker
We've tried that for decades.
00:38:39
Speaker
Honestly, even going back to the old Thorndike-Dewey debates back in the day, back in the 30s, people were debating whether or not multiple choice tests could be valid measures of intelligence.
00:38:53
Speaker
Honestly, I don't think it's worth the effort.
00:38:55
Speaker
We have tried to make tests more fair, more equitable, make them make more sense, make them more culturally responsive, and it doesn't work.
Financial Interests in Learning Loss Narratives
00:39:03
Speaker
Because ultimately, you can't take an entire year's worth of learning.
00:39:07
Speaker
in entirely different contexts and entirely different communities and say, all right, take this test and then figure out whether or not they learned something.
00:39:15
Speaker
It doesn't make any sense.
00:39:16
Speaker
There are other ways of measuring student growth.
00:39:20
Speaker
I think both of us would probably advocate for portfolios or showcasing of learning.
00:39:24
Speaker
And you could have perhaps a state board that reviews those things to see if teachers are on the right track and if kids are still reading and writing.
00:39:32
Speaker
It's not like they're
00:39:33
Speaker
we should get rid of all oversight it's just that that oversight should be working in a different way to support learners and support teachers instead of paying them against each other in a competitive testing environment especially when well one uh the testing obviously uh it hurts students from low-income backgrounds because typically funding is attached to this and students from low-income backgrounds face so many different challenges that they typically don't do well in standardized testing uh but two
00:40:02
Speaker
The overall narrative here, if we were to really go back, is that these tests are rooted in the idea of racist systems and practices.
00:40:10
Speaker
Like they're doing what they were designed to do.
00:40:12
Speaker
The whole idea of meritocracy through a multiple choice test, through the SAT and its origins,
00:40:19
Speaker
was designed to keep students of color and poor students out of higher education and out of potential opportunities for them.
00:40:28
Speaker
And that's documented.
00:40:29
Speaker
That's that's evidenced.
00:40:30
Speaker
That is a real thing.
00:40:32
Speaker
We know this.
00:40:33
Speaker
That's in the history.
00:40:34
Speaker
I mean, yeah, like it goes back, I mean, to Alfred Binet via Carl Brigham, who, I mean, was a eugenicist at his core.
00:40:46
Speaker
And Alfred Binet, you know, of course, with those IQ tests and those military proficiency exams around the turn of the century, too.
00:40:55
Speaker
And then, of course, lockstep with that eugenics movement with, you know, the development of racist science that validated in their minds Jim Crow and segregation and every other discriminatory policy imaginable.
00:41:11
Speaker
And, you know, at the risk of of Poe's law here, right, those things got exported to certain countries and they abused them in horrific ways, of course, in the 1930s and 40s.
00:41:22
Speaker
And
00:41:24
Speaker
We still use those exams at the same time.
00:41:27
Speaker
Right.
00:41:27
Speaker
They were they were deliberately designed to keep, you know, Jews out of Ivy League schools back in back in the day.
00:41:34
Speaker
And in the 1950s and 60s and stuff, there actually was like a huge as as desegregation led to more college entrance and stuff for for, you know, people of color in marginalized communities.
00:41:48
Speaker
The score percent, the score averages went down, right, as people from disproportionately underserved communities took these exams.
00:41:58
Speaker
And, right, this led to a little crisis in the SAT and in the college board, and like led to recentering the
00:42:08
Speaker
has led to a lot of things over the years to basically, right, again, from their perspective, if it to juke the stats, so you can keep the certain groups of people out under the more objective basis than just discriminated against them based on race, religion, socioeconomic status, but
00:42:28
Speaker
We know that these things are just proxies for all of those.
00:42:31
Speaker
One of the, I don't know if it's a Jack Schneider quote, something that he maybe tweeted out sometime during the pandemic, but he says like, well, we know if the argument is that we need these standardized test data to know which communities are underserved so that way we can better serve them.
00:42:48
Speaker
We already have that information.
00:42:49
Speaker
It's called their zip code.
00:42:51
Speaker
And there's no new information that these tests can provide that isn't already delivered somewhere else.
00:42:57
Speaker
So save the money that we would be having on be using to what, like take the tests, analyze the tests, make these things actionable, buy the accelerated learning software, and just give it to schools, right?
00:43:10
Speaker
To serve, to provide free lunches and breakfasts and meals and to...
00:43:17
Speaker
make before and after school programs and to build better schools and better conditions, like build air conditioning, build, right, the things that are going to serve them in this pandemic.
00:43:30
Speaker
So, yeah, so there is like a huge controversial history with that, that for some reason just gets thrown down the memory hole and we never really think about it.
00:43:41
Speaker
I mean, it goes back to even like picking up like a Jonathan Kozl book from the 70s, where we see this today.
00:43:49
Speaker
Anyone who's even vaguely familiar with public education in the United States knows that if you go to a school that typically serves students of low socioeconomic status, those schools tend to be more prison-like.
00:44:02
Speaker
They are hyper-vigilant about preparing students for tests.
00:44:06
Speaker
They tend to be more worksheet-driven, more lockstep.
00:44:10
Speaker
They sadly do things like lock the bathrooms and make you be silent in the hallways.
00:44:14
Speaker
They're super gross and dehumanizing.
00:44:17
Speaker
Meanwhile, a lot of politicians and some rich folks send their kids to schools where they do whatever they want, and there's a public, beautiful library, and everything's experiential, and
00:44:30
Speaker
their test scores still end up being higher
Stereotype Threat & Educational Expectations
00:44:33
Speaker
than the students who spent the entire year just preparing for that test.
00:44:37
Speaker
Because ultimately, test preparation and the way that we do test prep and the way that we focus on tests is a better indicator of how much wealth and how much access you have to, I guess, like a quality living circumstance.
00:44:49
Speaker
I don't know if that's the best way to put it, but I think you know what I'm saying.
00:44:52
Speaker
And I think the best way to highlight that, it's in our booklet we wrote about on page 19, is what happens if you get rid of the test?
00:45:00
Speaker
As of the 21, 22 school year, more than 1,500 U.S. four-year colleges have
00:45:05
Speaker
are not going to require ACT and SAT results.
00:45:08
Speaker
That's a lot.
00:45:08
Speaker
That's 65% of all four-year institutions in the United States.
00:45:12
Speaker
Because during the pandemic, people dropped tests and society did not collapse.
00:45:19
Speaker
And what's interesting to me is that in 2018, so this was before the pandemic, this was study, there was a research study that went out.
00:45:27
Speaker
And they looked at certain colleges and those that were test optional and they saw, well, what ramifications does this have if schools go test optional?
00:45:37
Speaker
To be clear, test optional means you can still submit test scores, but you can choose not to submit test scores.
00:45:45
Speaker
And I'll get to some caveats here in a second.
00:45:47
Speaker
But in a study, a fourth of all applicants didn't submit any scores.
00:45:51
Speaker
And as a result, obviously more people applied.
00:45:53
Speaker
However, interestingly enough, you saw higher numbers of Black and Latinx students apply and be admitted.
00:46:00
Speaker
What they found was that students of color were more likely not to submit scores.
00:46:06
Speaker
So 35% of black students did not submit scores in comparison to 18% of white students.
00:46:11
Speaker
And as you saw higher numbers of black and Latinx students be admitted, first-year grades were slightly lower for those who did not submit scores, but everyone graduated at the exact same rate or higher rates than their test-submitting peers.
00:46:25
Speaker
As in over time, it all equated out in the graduation rate
00:46:29
Speaker
was roughly the same or higher than the folks that did submit test scores.
00:46:34
Speaker
There is a caveat because test optional still doesn't work perfectly because those that submit good test scores are still given preferential treatment over those who choose not to submit test scores.
00:46:45
Speaker
But I think in practice, this shows that, I mean, if I was a college admissions counselor, I'd be like, well, what the heck?
00:46:52
Speaker
If students can get in and have the exact same graduation rate or higher graduation rate by not submitting a score, what does that mean for the test scores in general?
00:47:01
Speaker
Or a better question.
00:47:02
Speaker
Last year, during the pandemic, a ton of schools were test optional.
00:47:07
Speaker
If all of those schools and all of those students ultimately end up having the same graduation rate or higher and perform the same as or better their peers, why do any of this?
00:47:17
Speaker
What's the purpose of all this?
00:47:20
Speaker
It doesn't make any sense to me.
00:47:22
Speaker
I think especially if you consider...
00:47:24
Speaker
You know, we focus so much on that admissions process, you know, and getting kids to, you know, to the drive out to the parking lot at the college campus.
00:47:36
Speaker
And then we forget about it and we don't care.
00:47:38
Speaker
We don't realize, like, to what extent there actually is like a dropout crisis anyway.
00:47:44
Speaker
And if you look at the reasons that cause students to drop out, you
00:47:49
Speaker
it's overwhelmingly not because they, it's not because of scores.
00:47:56
Speaker
It's not because they didn't have background knowledge.
00:47:57
Speaker
It's not because, you know, they weren't well-prepared or whatever.
00:48:01
Speaker
It's because of cost.
00:48:03
Speaker
It's because it's expensive.
00:48:04
Speaker
It's because they have to, you know, take care of family members.
00:48:09
Speaker
So it's really about like those community support aspects too.
00:48:12
Speaker
So again, like the test score as college prep,
00:48:16
Speaker
conversation is like another non sequitur like squirrel, because then it gets us focusing on content and standards and, you know, the things that we need to micromanage in the classroom rather than like building an off ramp from kids to move from one environment successfully into another and then stay there, right?
00:48:38
Speaker
And be successful in those environments.
00:48:41
Speaker
One thing that's really interesting that I wrote about just pre-pandemic, this was like early March 2020, while this was all coming out, was about Grexit.
00:48:51
Speaker
And this was the GRE exit, where increasingly schools, particularly in what we call the hard sciences,
00:49:00
Speaker
like physics and some of those things too, they were going through hundreds of programs had been dropping GRE requirements.
00:49:11
Speaker
And what they found was that GRE didn't correlate at all to the quality and the success and the completion of their doctoral programs in these areas, but that GRE requirements actually did select against underrepresented groups.
00:49:29
Speaker
So it's like, okay, there's no correlation between the quality, but it keeps people from underrepresented groups out.
00:49:35
Speaker
What function does it serve other than like as a huge gatekeeper to, you know, people who traditionally have not been able to be entered into those programs?
00:49:46
Speaker
It just perpetuates those same biases.
Impact of Learning Loss Narrative on Student Perceptions
00:49:49
Speaker
It's almost like there is a system in the United States that could be analyzed about how different legal practices and practices of testing and meritocracy exclude certain groups intentionally.
00:50:00
Speaker
I wish there was a thing that folks learned about that dealt with
00:50:04
Speaker
You should develop a theory around that, perhaps.
00:50:07
Speaker
Maybe about critical race or something.
00:50:09
Speaker
I don't know.
00:50:10
Speaker
Speaking of research, too, I'm not going to go through this whole page.
00:50:14
Speaker
We have a page in the handbook that goes through just the amount of research that does not support standardized testing.
00:50:20
Speaker
There are studies that come from the 80s, the 90s, 2000s, 2010s, this year, about standardized testing.
00:50:27
Speaker
And I think almost every educator knows some of these.
00:50:30
Speaker
But I did want to highlight a few just because I forget
00:50:33
Speaker
just how big of an impact it is.
00:50:35
Speaker
Like there's obvious things that I just like, oh, right, this is a thing.
00:50:38
Speaker
We should probably care about this.
00:50:40
Speaker
Like the fact that our high school does not have a gym class because of a focus on college readiness.
00:50:45
Speaker
The students just don't have gym.
00:50:47
Speaker
So they don't, or phys ed or any kind of outdoor activity outside of what we integrate into our own classrooms.
00:50:54
Speaker
The fact that a lot of teachers will do like the quote unquote real curriculum after the test occurs.
00:51:01
Speaker
All the fun stuff happens after the test.
00:51:04
Speaker
The fact that educators and students just face so much pressure from this.
00:51:08
Speaker
You see a lot of teachers getting in a serious amount of trouble because they're worried that they're going to lose their jobs or have a salary cut.
00:51:15
Speaker
So they manipulate test scores.
00:51:17
Speaker
I don't understand, like, why in the world would you develop a system where people feel like they need to cheat when it's just a measure of their knowledge?
00:51:24
Speaker
It's not meant to be competitive.
00:51:26
Speaker
The fact that the results don't add up, we'll talk about that here in just a second with a little more than a lot of narrative in the McKinsey Report.
00:51:33
Speaker
uh engagement uh it is linked to pushing students out of school it's hyper boring no one likes doing test prep there are very few students who enjoy taking tests um and if they enjoy taking tests they can still take them but there's different ways of doing that they can get really good at trivia uh enjoy it there um the equity issue i think we've exhausted that conversation just
00:51:57
Speaker
The way that this selects against students of color is overwhelming.
00:52:02
Speaker
The data is so clear that it's disgusting to me that the conversation even has to occur.
00:52:08
Speaker
It's so obvious.
00:52:08
Speaker
And finally, just when you focus on preparing for the test, you lose out on so much.
00:52:15
Speaker
I think about all the different times where students have wanted to explore something that I thought was super cool, even in design, because I have a standardized test in design.
00:52:25
Speaker
And I'm like, well, I'd really like to do that.
00:52:27
Speaker
But if I focus on that right now, that's going to take a month.
00:52:29
Speaker
And there's no way you'll be able to get that part of that test done.
00:52:32
Speaker
Like we really do have to cover this to make sure that test is there.
00:52:35
Speaker
And always in the back of your mind, you can't lead a classroom that is truly based off of student interest.
00:52:40
Speaker
If constantly you have to shift and pull towards meeting ultra specific narratives and ultra specific questions that will appear on a test.
00:52:48
Speaker
It's not that the tests are never broad enough to encompass just generalized learning during the course of a year.
00:52:54
Speaker
Yeah, right when you've gotten past the initial sort of bite, right as kids begin to grasp a concept or begin to get excited about it and want to dig and dive in as deep as they can on that thing, you're moving right along.
00:53:08
Speaker
So even the concept of learning loss kind of takes...
00:53:12
Speaker
the content coverage narrative of this thing too, right?
00:53:16
Speaker
And says that like, you know, we can break these into these discrete bits of information that need to be delivered on X, Y, and Z day.
00:53:23
Speaker
And so if you miss so many days, right, you're missing these discrete bits of information instead of, right, losing out on the process, right?
00:53:32
Speaker
And not being able to practice, right?
00:53:34
Speaker
You know, the kinds of thinking skills that, you know, otherwise competent, successful adults might be able to practice.
Conclusion & Preview of Next Episode
00:53:41
Speaker
No, you're missing the discrete bits of information that are going to be used against you and your and your schooling institutions later on.
00:53:49
Speaker
Exactly.
00:53:50
Speaker
So I realized this is going to end up being a real long podcast, but that's all right.
00:53:55
Speaker
Because I honestly think that every part is just so hyper valid.
00:53:58
Speaker
So hopefully people are right.
00:53:59
Speaker
It's pause and take a break through this thing.
00:54:01
Speaker
Because this next part to me is the most interesting.
00:54:05
Speaker
I feel like everything that we've covered up until this point, if someone's been involved in education, they probably realize, yeah, this is a problem.
00:54:11
Speaker
And hopefully we're providing more evidence and more tools to communicate that.
00:54:16
Speaker
But the next section deals with the connection of the standardized testing industry and this new learning loss narrative, the one that we're currently involved in right now.
00:54:26
Speaker
This was the part that was done first before we did anything else.
00:54:30
Speaker
This one was mostly me sitting down and it's the Charlie Day meme from Always Sunny.
00:54:38
Speaker
What's the woman's name?
00:54:41
Speaker
Who is Pepe Silvia?
00:54:42
Speaker
Yeah, who is Pepe Silvia?
00:54:43
Speaker
It's that meme with a conspiracy theory.
00:54:46
Speaker
So I literally made a graphic showcasing all of the weird connections between the testing industry, testing companies, and
00:54:57
Speaker
and the reports that highlight this idea of learning loss.
00:55:01
Speaker
And here's the gist of it.
00:55:03
Speaker
So you have like Pearson, Savas, College Board, NWEA, Curriculum Associates, iReady, Illuminate.
00:55:11
Speaker
These are all testing companies.
00:55:13
Speaker
Some of them are huge scare quotes, nonprofits, and some of them are corporations.
00:55:19
Speaker
that generate an absurd amount of revenue.
00:55:22
Speaker
College board alone generates a billion dollars a year, over a billion.
00:55:26
Speaker
Technically, you could make a billion dollars and still be a nonprofit somehow.
00:55:30
Speaker
I'm not really sure how that works.
00:55:32
Speaker
And pay your executives a lot of money, FYI.
00:55:36
Speaker
You have just an absurd amount of money.
00:55:39
Speaker
And pretty much every single one of these organizations sells services that address the concept of learning loss.
00:55:48
Speaker
So obviously, all these testing companies offer textbooks, resources, et cetera, that you pay for in order to do better at their test.
00:55:55
Speaker
But now all of these testing companies are...
00:55:58
Speaker
selling explicit services to address learning loss.
00:56:02
Speaker
That this is this major problem where, hey, you need to buy this special curriculum and do this at the beginning of the year.
00:56:06
Speaker
Our school has done this.
00:56:09
Speaker
In fact, the federal government has required
00:56:11
Speaker
that 20% of school budgets be dedicated towards addressing what was lost in the COVID year, which is highly connected to this learning loss narrative.
00:56:19
Speaker
So I would not be surprised that pretty much everyone listening has had to purchase some kind of testing software or do increased testing as a result of this learning loss narrative.
00:56:29
Speaker
So they're all offering these services, making them even more money.
00:56:32
Speaker
But where this comes from, so like the way that we know that we need to address learning loss is mostly through three reports, the big one being the McKinsey
00:56:44
Speaker
So tying it back together with that global consulting firm, McKinsey.
00:56:48
Speaker
McKinsey published a report in 2020.
00:56:50
Speaker
They actually established multiple reports now saying that, hey, we have this major problem as a result of students leaving school during the pandemic year and schools shutting down and going virtual, even though they don't always say that.
00:57:06
Speaker
Students have a lot of academic needs.
00:57:08
Speaker
So we need to make sure that we address this.
00:57:10
Speaker
So McKinsey pulls data from Illuminate Education, iReady, and a couple others to say, hey, look, test scores are going down.
00:57:19
Speaker
Then all of the testing companies point to the McKinsey report and say, man, this is terrible.
00:57:24
Speaker
Look at this.
00:57:25
Speaker
McKinsey says that all the test scores are going down.
00:57:27
Speaker
They analyzed all of our data.
00:57:28
Speaker
So therefore, you need to spend $25,000 this year to purchase something to address this.
00:57:34
Speaker
But here's where it gets interesting.
00:57:38
Speaker
The actual data, even at taken at face value, if we ignore the fact that the test itself doesn't mean anything, that it's irrelevant, that the questions that are being asked are absurd,
00:57:51
Speaker
The data doesn't add up.
00:57:53
Speaker
So we dove into the actual reports.
00:57:57
Speaker
We looked at what these reports said, how much learning was lost, quote unquote.
00:58:02
Speaker
And here's a few examples.
00:58:03
Speaker
So you have the McKinsey report, the Annenberg report, and the Illuminate Education report.
00:58:09
Speaker
Often in articles, you'll see the McKinsey report cited.
00:58:12
Speaker
And now they started citing each other.
00:58:14
Speaker
So like Annenberg will cite McKinsey, McKinsey will cite Illuminate, et cetera.
00:58:18
Speaker
Those are the only three major ones to our knowledge.
00:58:21
Speaker
In the Illuminate report, there is this chart, which is a screenshot.
00:58:25
Speaker
It's on page, well, it's in the learning loss section, which is actually just on the page, but I think it's page 24.
00:58:30
Speaker
Here there is a figure which shows grade one A math scores, which is an adaptive test that shows students math level.
00:58:40
Speaker
The y-axis is the predicted A math score.
00:58:43
Speaker
The x-axis is a student's free and reduced lunch amount.
00:58:46
Speaker
And the lines represent the 2018, 2019.
00:58:47
Speaker
And then the other one is the 2019 or 2020 school year.
00:58:52
Speaker
So for example, if you had a student who was on entirely free to reduce lunch in an urban school environment, and they received a predicted score of, let's say, 197, the previous year they received a 200, which if anyone is having trouble visualizing that, that's a three-point difference.
00:59:09
Speaker
So from a quote-unquote normal school year, the score was 200, to the year where school shut down and a global pandemic shut us all down, their score was a 197.
00:59:22
Speaker
That, when you look at all the data on this entire report, is the largest gap.
00:59:29
Speaker
As in, the largest gap that is used on this entire report is three points.
00:59:35
Speaker
And that is three points on an over 200-point scale.
00:59:39
Speaker
So it's not even 3%.
00:59:42
Speaker
We're talking like minute percentage points.
00:59:44
Speaker
In fact, if you were using an adaptive test model, we're talking about probably one or two questions difference.
00:59:52
Speaker
There's a similar one in here where it's third grade aid reading.
00:59:56
Speaker
It's a wider scale.
00:59:57
Speaker
So the scale is actually 510 points roughly.
01:00:01
Speaker
There was a one to two point drop year to year.
01:00:05
Speaker
So one to two points on a roughly 510 point scale.
01:00:09
Speaker
That's it.
01:00:10
Speaker
And the most interesting part about this whole thing, this is the one that blew my mind.
01:00:15
Speaker
This is the one that I screenshot, I tossed in our Discord, I'm like, yo, can you believe what this is written in here?
01:00:20
Speaker
I'll just read it ad verbatim.
01:00:21
Speaker
So this company, which is looking at this data saying, look how terrible this is, look at this data.
01:00:27
Speaker
They also say like,
01:00:29
Speaker
Hey, we want to bring up the fact that MAP and STAR tests maybe didn't find the same thing that we did in their reports.
01:00:36
Speaker
They write, in the comparison with MAP math results, the drop in the medium national percentile is very similar to FastBridge results, with differences ranging from zero to three percentile points.
01:00:48
Speaker
It is worth noting that when analyzing learning loss in FastBridge by using changes in mean achievement scale scores across years, the estimated learning loss is consistently similar than the loss derived from ROI differences.
01:01:01
Speaker
This may imply that MAP results underestimate true learning loss.
01:01:05
Speaker
As in, they looked at another report from the map in the NWEA and saw they said that they might have not lost any points at all.
01:01:14
Speaker
So actually, they did some advanced formula wrong.
01:01:20
Speaker
And we found that actually there might have been a three point drop.
01:01:24
Speaker
And it's like, wait, what?
01:01:26
Speaker
Like this to me is so wild.
01:01:28
Speaker
Like it still riles me up.
01:01:30
Speaker
Like we're freaking out about three points on a 200 point scale, AKA one or two questions on an adaptive test.
01:01:39
Speaker
That's what all this money is about.
01:01:41
Speaker
This entire thing is rooted on this.
01:01:43
Speaker
This is the basis.
01:01:44
Speaker
What is amazing, if you're looking at that chart of the AMATH scores, it is so deceptively designed because that Y-axis, you would think that the spread in that Y-axis would be 50 points or something like that, right?
01:02:00
Speaker
But it goes from 190 to 195 to 200, 205, and 210 all
01:02:06
Speaker
over about the space of two inches, so that way they can get the little decline on their curve, so you can actually visualize that.
01:02:15
Speaker
If that was a scale that went from zero to 210, it would be imperceptible, imperceptible decline in that thing.
01:02:23
Speaker
Like if we were talking about getting a rocket to the moon, right?
01:02:29
Speaker
Like that 3% would be probably meaningful.
01:02:33
Speaker
That would be valid.
01:02:34
Speaker
That would be like reliable information that we would need to act to correct.
01:02:38
Speaker
But given all of the other doubts about the utility of those three points and what those one to two questions are asking, how they're asking them, are they things that are worth responding to in the first place?
01:02:54
Speaker
Since that is not an exact science, since that's very much a value-driven proposition, and oftentimes we don't have access to what those questions are, how they're phrased, what it is exactly that they're asking and in what way or what might be confusing for students, we just have to take it on face value from...
01:03:15
Speaker
from wherever, from Fastbridge, that this drop is not only meaningful, but it's important and worth acting on.
01:03:25
Speaker
And again, and I'm going to keep ringing this bell, right?
01:03:28
Speaker
The fact is that we're seeding, like we're giving up
01:03:33
Speaker
agency, we're giving up power to the people furthest away from the classrooms to make decisions on our behalf based on data that they control and that we have to pay for, but that they expect us in the classroom to enact changes and curriculum and assessments to answer the
01:03:54
Speaker
not to the kids in front of us, not to parents in the community, right?
01:03:58
Speaker
Not to the goals of, you know, the school district or whatever else, but to their test.
01:04:06
Speaker
In my opinion, this is intentionally hyper complex.
01:04:11
Speaker
I feel like this is all very much buried, both from the design of the graphs to how it's written, to how you find this data.
01:04:20
Speaker
Because if you open up this report and you read the summary,
01:04:24
Speaker
The very beginning of the Illuminate Education report starts talking about how we need to make up multiple months of lost time.
01:04:32
Speaker
They are translating this percentile drop to literally one to three months of learning, which is not rooted on anything.
01:04:41
Speaker
That is not based off an equation.
01:04:42
Speaker
That is not based off of some in-depth study.
01:04:45
Speaker
That is Illuminate Education's opinion of what they describe as negligible to moderate loss.
01:04:52
Speaker
Which, there's something very funny to me about the idea that one month of learning was considered negligible.
01:04:59
Speaker
Like to me, one month of learning is a lot if we were going to measure it in time, which is also absurd.
01:05:05
Speaker
Even if, again, we take the data at face value, you're telling me that in the face of a global pandemic that shut down schools where kids learned online and teachers were scrambling to figure out what the heck to do.
01:05:17
Speaker
I'm sure everyone remembers like March slash April when school shut down.
01:05:21
Speaker
You're like, what the heck is this?
01:05:24
Speaker
That in that whole time period, kids maybe missed one or two more questions.
01:05:28
Speaker
And that's the problem.
01:05:30
Speaker
That would be assumed.
01:05:32
Speaker
It doesn't necessarily mean that we need to be frantic.
01:05:35
Speaker
Speaking of data that is very questionable in how it's presented, I also wanted to call to attention the McKinsey report.
01:05:44
Speaker
which is if you read an article about learning loss, again, we've been joking about using like your control F and searching McKinsey, it'll be in there.
01:05:54
Speaker
Everyone references McKinsey about how terrible learning loss is.
01:05:58
Speaker
Their report pulls from the map, which is the one that most students take, the NWEA map.
01:06:04
Speaker
Essentially, they highlight this one graph in here that shows significant learning loss or risk for significant learning loss.
01:06:12
Speaker
And it's a chart that shows
01:06:14
Speaker
four different lines going across.
01:06:16
Speaker
And the lines are meant to predict a scenario where it's just normal learning, where students are just in class and then the chart goes up and then summer hits and it starts to fall a little bit.
01:06:29
Speaker
The second scenario is 52% learning growth through quality remote instruction.
01:06:37
Speaker
So someone who wrote this report was like, if you do high quality virtual instruction, then students learn at 52% starting in March, which again, that's not based off anything.
01:06:51
Speaker
That is just an assumption.
01:06:52
Speaker
It's an estimate, an estimation.
01:06:56
Speaker
So then on that little chart, it's a few points lower, it goes up a little bit and then goes back down.
01:07:01
Speaker
The third line is that no growth or loss occurs at all.
01:07:05
Speaker
So this is low quality remote instruction, as in you're treading water.
01:07:10
Speaker
Students aren't learning anything, but they're not losing anything either, which I'm assuming is like worksheets provided online.
01:07:16
Speaker
I don't know that these terms, high quality and low quality instruction are never defined.
01:07:21
Speaker
And then finally, no instruction occurs at all.
01:07:25
Speaker
So there is a line on here that says, hey, school's closed and they never reopened.
01:07:30
Speaker
And the line just goes straight down and no one learns anything.
01:07:32
Speaker
Ignoring the fact that that assumes that you learn everything in school.
01:07:37
Speaker
So much of this is just an entire assumption.
01:07:41
Speaker
Like who, why 52%?
01:07:42
Speaker
Why?
01:07:42
Speaker
Like where, why would we think that it's 52%?
01:07:45
Speaker
And again,
01:07:51
Speaker
Even taken at face value, I would argue that the vast majority of schools in the United States either taught, let's say, with high quality or low quality materials.
01:08:02
Speaker
I don't know what that means, again, virtually.
01:08:05
Speaker
But the difference is a 230 RIT score, which is like an adaptive score, learning occurring.
01:08:10
Speaker
We define what RIT scores is.
01:08:11
Speaker
I don't want to go into that.
01:08:14
Speaker
230.
01:08:15
Speaker
And the lowest one on here is like roughly a 219.
01:08:19
Speaker
So if we average that, let's say, or even did like a low average, at most they lost 10 points on a over 200 point scale because the RIT again is adaptive.
01:08:31
Speaker
We're talking again a difference between let's say three or four questions, again based off entirely unknown data.
01:08:38
Speaker
That is being translated into saying students lost multiple months of learning because they missed a few questions on the test that they didn't normally before.
01:08:47
Speaker
What is super interesting, too, is you're speaking of the data, of course, that goes from typical in-person and then looks at the various qualities of remote instruction, right?
01:08:57
Speaker
And that's within a 10-point range.
01:08:59
Speaker
But even the lowest one that assumes that there is no instruction, like a kid just falls off the grid, right?
01:09:06
Speaker
Shuts off their brain for what?
01:09:09
Speaker
Like 13 months is what this chart basically goes through, right?
01:09:12
Speaker
Or longer than that.
01:09:14
Speaker
even then it goes from what looks like about a 222 to like a 208.
01:09:18
Speaker
So again, assuming that you've just straight up been gone for a dozen months of instruction, your scores are gonna go down 15 points.
01:09:31
Speaker
And that I think, I don't wanna jump the gun on things, but it's like, man, if the only thing that we're doing in schools is valued by the way that it's measured on tests,
01:09:42
Speaker
That would, you know, this would be frightening in how little it actually impacts these scores.
01:09:49
Speaker
But as we begin to ask that question about like, well, what are we actually losing?
01:09:53
Speaker
Like, what function should schools serve in growing like little human beings?
01:09:59
Speaker
You know, is it the case that those things can be accurately distilled down into that assessment score?
01:10:06
Speaker
Or if you miss a month of school, right?
01:10:09
Speaker
Are you missing out on something else that is just not being captured?
01:10:13
Speaker
Right.
01:10:13
Speaker
That's the value of school.
01:10:14
Speaker
Those things are not being captured on on those assessment scores.
01:10:19
Speaker
And we haven't yet really figured out a way to and I'm not trying to give anyone any ideas, but evaluate.
01:10:26
Speaker
Right.
01:10:26
Speaker
Like kind of systematize those things.
01:10:28
Speaker
And this is this has been Susan Engel's point, you know, as a developmental psychologist.
01:10:35
Speaker
And former HRP board member for the quick plug, Dr. Susan Engel, she would make this point in a heartbeat, right?
01:10:41
Speaker
Like we have ways of evaluating curiosity, right?
01:10:45
Speaker
Of figuring out affective, you know, ways that students feel about school, right?
01:10:51
Speaker
And feel about their time in those buildings and how we can, how can we measure those qualitatively and quantitatively and use that data to improve those measures, right?
01:11:01
Speaker
But that's not data that the learning loss folks have access to.
01:11:04
Speaker
That's not anything that McKinsey cares about.
01:11:07
Speaker
All they're doing is looking through this very narrow slit at the tire functions of the eight hour school day plus or minus two hours on either end of that thing or more.
01:11:19
Speaker
And then saying, nope, 10 points on your map tests.
01:11:24
Speaker
using RIT scores.
01:11:26
Speaker
So yeah, like what are we actually losing in that time is going to have to be a conversation that we have as part of this too.
01:11:32
Speaker
Yeah.
01:11:32
Speaker
And the only other report I really wanted to draw data on because this was the funniest one to me is so McKinsey pulls from the NWEA data.
01:11:43
Speaker
NWEA, the map creators also have their own publication, which features the exact same data.
01:11:51
Speaker
It's where the chart comes from the McKinsey report.
01:11:53
Speaker
What's interesting about this one is the one that we link to basically shows like, again, like very, very, very minor differences in scores over time, basically the exact same data.
01:12:04
Speaker
But since we've published this, there have been updated charts.
01:12:08
Speaker
So a lot of these folks have walked back previous statements.
01:12:12
Speaker
They've also rebranded.
01:12:13
Speaker
So it's no longer learning loss.
01:12:15
Speaker
It's unfinished learning.
01:12:17
Speaker
which sounds a lot nicer, but it's really rooted on the exact same thing.
01:12:21
Speaker
The data didn't end up being nearly as bad as they predicted it was going to be.
01:12:24
Speaker
It actually was much higher.
01:12:25
Speaker
In some cases, we looked at one where the scores were higher than the previous year.
01:12:31
Speaker
Even in the state of a global pandemic, students actually had higher test scores than they did the previous year, which really makes you question, why are we focused so much on this?
01:12:41
Speaker
The whole point that we're trying to make here is that there's much ado about nothing.
01:12:46
Speaker
we're talking about differences that are so small, but yet have massive ramifications because of how this has been both, um, narratively driven into a sense propagandized, um,
01:13:00
Speaker
The Department of Education has been highly rooted in this.
01:13:03
Speaker
Again, not to sound conspiratorial, but most of these testing companies lobby the government and say, hey, this is an equity problem.
01:13:11
Speaker
Students need to do better.
01:13:13
Speaker
The achievement gap is getting worse.
01:13:15
Speaker
And it's sold on this idea of we need to help students that are most vulnerable.
01:13:20
Speaker
But as we know, as educators, focusing on standardized testing does not help students who are most vulnerable, that it actually hurts students that are there.
01:13:30
Speaker
And we have a large research database that demonstrates that, in my opinion.
01:13:35
Speaker
So earlier, I had mentioned Operation Reverse the Loss, which is the Department of Education's new initiatives to invest in solutions to the learning loss problem.
01:13:48
Speaker
Essentially, in order to get money through the American Rescue Act, states have to invest 20% of their funding to explicitly address issues of academic loss.
01:13:59
Speaker
And you can find the link in our handbook.
01:14:02
Speaker
It's from Future Ed.
01:14:05
Speaker
They've documented what every single state has done and where their money has gone.
01:14:09
Speaker
And you'll find things in there like summer reading and math camps, intensive programs, a lot of adaptive learning software, promoting concepts like learning acceleration, creating new screening measures, so taking even more tests.
01:14:24
Speaker
I know we're taking an additional math test this year, so now the kids have to take even more tests.
01:14:29
Speaker
PD programs focused on addressing learning loss.
01:14:32
Speaker
So although there are states investing in social-emotional well-being and trauma-formed education,
01:14:39
Speaker
But unsurprisingly, it seems like the vast majority of funding is going towards this explicit problem.
01:14:44
Speaker
And I can't say this enough because I feel like I'm losing my mind, but we're talking about one, two, or three questions on a test.
01:14:52
Speaker
It would maybe be different if we were saying like these kids failed half the test and it's like a national panic, but it's not.
01:14:59
Speaker
This is a very minor problem.
01:15:01
Speaker
And I think honestly, it's just rude and making more money for testing companies.
01:15:04
Speaker
So basically, I think what we're getting at as Human Restoration Project when it gets to this is one, we're connecting the financial industry to the learning loss narrative.
01:15:14
Speaker
We are trying to provide educators with the knowledge and the tools to recognize that they are not going crazy.
01:15:21
Speaker
This is not rooted in fact.
01:15:24
Speaker
This is not nearly as meaningful as we say it is.
01:15:27
Speaker
The data itself that we're pulling from doesn't even make any sense.
01:15:31
Speaker
But when we're focused on things that were lost during the pandemic, I mean, at the time of writing this report, we were at 600,000 lives.
01:15:39
Speaker
I believe we're much higher now.
01:15:41
Speaker
Due to COVID, people have died.
01:15:43
Speaker
You know, that affects personal relationships, our mental health.
01:15:47
Speaker
Learning online is...
01:15:48
Speaker
for some an improvement, but for many, it was a struggle, especially when it comes to social connections and all those things that kids like to do.
01:15:57
Speaker
We did our 100 Days of Conversations initiative where we collected and we're about to publish the data of I think 107 different conversations with young people and adults.
01:16:08
Speaker
And almost universally, students said, hey, what I didn't like about the pandemic was I didn't get to go to school dances.
01:16:14
Speaker
I didn't get to hang out with my friends.
01:16:16
Speaker
I didn't get to
01:16:17
Speaker
Like take my driver's test to get my license.
01:16:20
Speaker
I didn't get to eat out anymore.
01:16:23
Speaker
Like things that are the social experience of being someone in school.
01:16:29
Speaker
That is the thing that students struggled with.
01:16:32
Speaker
they didn't really talk about, hey, it was really hard to take the test this year, or I really don't get math anymore.
01:16:40
Speaker
Math is just an enigma to me.
01:16:42
Speaker
It's probably the same as it always has been.
01:16:45
Speaker
So if we are focusing instead on, at the start of the school year, on saying, oh my goodness, test scores have gone down.
01:16:52
Speaker
We need to fix this learning loss.
01:16:54
Speaker
Here's the PD I just did.
01:16:55
Speaker
Here's all this test prep material.
01:16:56
Speaker
We are ironically,
01:16:58
Speaker
causing the same problem that students are saying that they have, which is loss of social connection, because those activities destroy that.
01:17:07
Speaker
Yeah, I think one thing that I have definitely noticed in my students this year, especially my seniors, you know, my sophomores, I don't know if it's a resilience thing or whatever, they've just, they've been as disrupted, right?
01:17:21
Speaker
But my seniors, I think, heading into their senior year, they're
01:17:24
Speaker
they're going to be grateful to be done with the herky jerky, you know, all of this stuff.
01:17:29
Speaker
They're, they're kind of, they're kind of fed up with it.
01:17:31
Speaker
And, and the thing that I see in them is not like, like they don't know as much, right.
01:17:36
Speaker
Or they weren't able to cover as much in their U S history class last year, and now they're unprepared, but their perspective on how schools should be and their, their role as learners, I think has, has been changed for the worse.
01:17:51
Speaker
I think that this,
01:17:52
Speaker
The class of seniors that I have, I love them, but they have a very transactional view of how education should work, in part because we really failed to invest in, you know, call it high quality distance learning or digital learning or whatever.
01:18:08
Speaker
We've had digital learnings forever, right?
01:18:11
Speaker
And it has been, it can be high quality if you pair it with
01:18:16
Speaker
A coherent digital pedagogy.
01:18:18
Speaker
And that was the failure.
01:18:19
Speaker
Okay, we tried to, we tried, we tried to reinvent school online, and then bring it into kids bedrooms, you know, in those traditional power structures, those, you know, those behaviorist systems, etc, just just didn't translate.
01:18:36
Speaker
And so the ways that we tried to mitigate it not working was to double down on the transactional kind of model, right?
01:18:43
Speaker
Like school as this checklist, you check in, you get these points, you do whatever.
01:18:49
Speaker
And kids, I think they still have that view a little bit because we didn't invest in the digital pedagogy that meets them in their bedroom and sees that as an opportunity, right?
01:18:59
Speaker
And doesn't see it as a barrier to...
01:19:02
Speaker
to schooling, um, but views, you know, what can we do today with you that moves the ball forward with something in your life?
01:19:10
Speaker
Right.
01:19:11
Speaker
And then maybe how can we make math a part of that?
01:19:13
Speaker
Right.
01:19:14
Speaker
But start with students first.
01:19:15
Speaker
Now, when I, I think I was at least marginally successful in doing this with my remote students who now I see in person in the building, uh, now this year and, uh,
01:19:27
Speaker
It was pretty incredible because it's the first time I've been able to see a lot of them face to face.
01:19:31
Speaker
But, you know, those are some of the strongest relationships that I was able to build last year because the in-person environment came with so many barriers and hurdles and sanitation policies and so much anxiety around it already that.
01:19:47
Speaker
My online cohort just didn't have.
01:19:49
Speaker
So those are kids that feel free, even now, coming to me in person and talking about things, talking about their experiences last year, really as a meaningful and impactful thing and how we grew as a community of people all experiencing this trauma together.
01:20:05
Speaker
And that's a thing, too, that I think we failed to really meet in this transition back into school is that that learning loss narrative can end up being a self-fulfilling prophecy.
01:20:15
Speaker
If all students end up hearing is that, you know, that that online learning was bad, that I didn't learn what I didn't do what I should have done, that, you know, school failed, that et cetera, et cetera, et cetera.
01:20:28
Speaker
Like they they adapt that.
01:20:31
Speaker
But again, if we had flipped that to be something more positive and we could say like, hey, how did we survive during this time when a lot of people did not?
01:20:42
Speaker
How can we come back and again, borrow Young Zhao's framing of this, like how can we take the lessons of the pandemic and build back better instead of beat kids over the head with this deficit narrative from day one?
01:20:57
Speaker
I mean, the purpose of this is to contextualize this, to understand what's going on, because there is an ability to push back against it.
01:21:07
Speaker
We'll talk about the how in a lot more detail in the next podcast, but understanding this stuff gives you the tools to recognize that you can push back and there is a plausible argument on why we're doing this.
01:21:19
Speaker
It's not just a gut instinct.
01:21:21
Speaker
It's real, just like all the other things involving progressive education, like ungrading and restorative justice, et cetera.
01:21:27
Speaker
It's not just it feels bad, there is something actually wrong with the current system.
01:21:32
Speaker
That's what we're trying to contextualize here.
01:21:34
Speaker
So the last two things I wanted to highlight is how the narrative and framing can affect how we treat students and how they feel.
01:21:43
Speaker
The first one I wanted to call out
01:21:45
Speaker
is the stereotype threat, which is typically attributed to Claude Steele.
01:21:49
Speaker
He wrote the book Whistling Vivaldi, which is an incredible book.
01:21:53
Speaker
The stereotype threat is essentially the risk of confirming your own negative stereotype because you're perceived to have a lack of abilities.
01:22:01
Speaker
So, for example, one of the first studies in that book talks about that they brought in many students, I think it's 54 students, with strong ACT, SAT math scores.
01:22:12
Speaker
They were in really high calculus courses.
01:22:14
Speaker
These are students who test well in math.
01:22:17
Speaker
That is their skill set.
01:22:19
Speaker
They had two groups.
01:22:21
Speaker
They had a group that they told the students, hey, this might show gender differences in math ability.
01:22:27
Speaker
And then they told the other group, hey, this test is explicitly designed not to show gender differences and math ability.
01:22:34
Speaker
And for students who were told, hey, there might be a gender difference here, overwhelmingly, male students did better.
01:22:44
Speaker
But when the group was told explicitly, hey, this test does not show any gender differences designed not to, it was roughly the same between men and women.
01:22:55
Speaker
And they repeated this experiment over and over and came to the conclusion that female students, when told of this measured gender difference, there's a perceived cultural norm that men are better at some subjects than women.
01:23:09
Speaker
Which is obviously not true.
01:23:11
Speaker
But when that cultural norm is kind of pitted against you, you start to feel a lot of anxiety, you feel like you might need to overthink questions, you feel like you're not gonna perform well, you feel like, oh, this test might be like not designed for me, etc.
01:23:25
Speaker
And you psychologically do worse.
01:23:28
Speaker
And that's shown when you get rid of that explicit narrative.
01:23:32
Speaker
They did the exact same thing.
01:23:34
Speaker
This book is so interesting, by the way.
01:23:36
Speaker
I love this book.
01:23:37
Speaker
They did the exact same thing with white and black athletes in terms of this concept of sports intelligence through golf.
01:23:44
Speaker
I'll just briefly summarize this one.
01:23:46
Speaker
They basically brought in different golf players, like high-level golf players who were black and white.
01:23:52
Speaker
And they gave them different ways of taking this test.
01:23:57
Speaker
And they told like one group, hey, this measures natural ability.
01:24:01
Speaker
Hey, this measures sports intelligence.
01:24:03
Speaker
They had one group take a like a demographics question before they did the test.
01:24:10
Speaker
They had another group that it was just like general sports performance and they didn't collect the demographic data until after.
01:24:17
Speaker
people mirrored the cultural stereotypes.
01:24:19
Speaker
So like when, for example, they were told that this measures sports intelligence, white athletes had better scores than black athletes because of like the perceived verbiage of sports intelligence, which is defined as the ability to think strategically during an athletic performance.
01:24:38
Speaker
Or, for example, when folks were given a survey beforehand about demographics and being a golf player, and when people took that demographic quiz beforehand, white athletes did better.
01:24:49
Speaker
When people took the demographics question afterwards, it was about the same.
01:24:54
Speaker
It's all based around this idea, this threat of, hey, if I know that this stereotype exists about me and I feel like it's going to be a thing, then I might underperform because of that anxiety and narrative and centering that idea.
01:25:12
Speaker
This connects to the learning loss narrative because in media framing of learning loss, there's a stereotype threat against those of marginalized backgrounds.
01:25:23
Speaker
This whole narrative is primarily centered on schools that already were struggling through school report cards, etc.
01:25:30
Speaker
And that typically affects those of marginalized backgrounds the most.
01:25:35
Speaker
teaching your class and you are centering this idea of learning loss and saying about how much was lost and you constantly reference this and you say hey you know we got to make sure we improve test scores this year they weren't great uh everybody lost learning you might
01:25:51
Speaker
I guess inexplicably or unintentionally end up making those students perform worse, especially if you bring in, I hope you wouldn't do this, but if you were bringing in the data itself, it's going to talk about marginalized communities.
01:26:06
Speaker
It's going to talk about absenteeism rates.
01:26:08
Speaker
It's going to talk about inequality and inequity.
01:26:12
Speaker
So recognizing the fact that narrative can have threats.
Teacher Expectations & Student Performance
01:26:16
Speaker
The other one that we highlighted was the Pygmalion effect, or also called the Rosenthal effect, which is basically where they took a group of elementary school teachers and they, it was through Harvard University, these Harvard University researchers came in.
01:26:31
Speaker
They told these elementary school teachers, hey, we designed this cool new test, and it's going to identify who the spurters are.
01:26:38
Speaker
So the students that are really go-getters, that are academically talented, socially talented, these are just great kids.
01:26:44
Speaker
And they had all these kids take this fake test, and the researchers took all this data, and they chose...
01:26:52
Speaker
20% of those students at random to be spurgers.
01:26:55
Speaker
There was absolutely no data involved in this.
01:26:57
Speaker
They are just like, these 20% are spurgers.
01:27:00
Speaker
And they took it back to the teachers and said, yeah, these are the spurgers.
01:27:03
Speaker
These are the successful ones.
01:27:04
Speaker
These guys are great.
01:27:05
Speaker
They did so well on the test.
01:27:08
Speaker
And they checked in with these teachers throughout the year.
01:27:11
Speaker
There were two things that they found.
01:27:13
Speaker
First off, the students that were identified as Spergers, especially at younger age levels, like first grade, second grade, drastically outperformed students on tests, like unbelievably doubled their score on tests versus students who are identified as non-Spergers.
01:27:31
Speaker
And they did narrative.
01:27:33
Speaker
They would go around and talk to the teachers and say, what's the behavior like of these Spergers?
01:27:38
Speaker
And the teachers were talking about how they were going to be more successful.
01:27:41
Speaker
They were curious.
01:27:42
Speaker
They were interesting people and just how awesome they were.
01:27:45
Speaker
And then the researchers would ask them about the students who were not Sprueters.
01:27:50
Speaker
And then they would say, oh, you know, that kid, you know, I don't know if they're going to make it.
01:27:54
Speaker
They're not really that great.
01:27:55
Speaker
And interestingly enough, if one of the non-spurs was doing really well, they wouldn't describe them in positive terms.
01:28:01
Speaker
They would say like, oh, it was really unexpected.
01:28:03
Speaker
They're really unique.
01:28:05
Speaker
It's weird how they're able to do this without being identified as such.
01:28:09
Speaker
It was almost like a negative.
01:28:11
Speaker
And the researchers attributed that to the fact that when teachers saw that these certain students were spurters, they maybe gave them more one-on-one attention.
01:28:19
Speaker
They were a little more friendly.
01:28:21
Speaker
They were more affirming.
01:28:24
Speaker
They felt like maybe the reason why younger learners were more susceptible is that when you're younger, you tend to be a little more like malleable.
Framing Educational Success
01:28:30
Speaker
So it's like, hey, the teacher's paying attention.
01:28:31
Speaker
attention to me, this is cool.
01:28:33
Speaker
And because you got paid more attention to you tend to learn more for the test and you tend to be like a happier kid because the teachers pay attention to you and working with you and doing cool stuff.
01:28:42
Speaker
Whereas everyone else feels left out.
01:28:45
Speaker
That to me is just so fascinating because there's so many different ways that you can go with that.
01:28:49
Speaker
There is such an interesting, again, like that affective domain of this thing, because in the bottom of that page 31, it says, the researchers asked teachers about the behavior of their pupils.
01:28:58
Speaker
They consistently noted the spurters had a better chance of being successful and happy later in life, as well as being more curious and interesting than other children.
01:29:08
Speaker
So that is, it's kind of interesting to pit at the beginning of this 90-minute episode.
01:29:16
Speaker
I had mentioned, you know, the apocalyptic tones in which, you know, the New York Times, like the Fordham Institute or whatever talks about the impact of learning loss on education.
01:29:27
Speaker
incomes over the lifetime of the child and of the hit in billions of dollars to national GDP, et cetera.
01:29:36
Speaker
And that's based off of extrapolation from two or three points in the score data that we've been talking about, et cetera.
01:29:44
Speaker
But here it's the case too, that's like just the way that we treat kids, right?
01:29:49
Speaker
can have a immeasurable impact on what being successful and happy, more curious and interesting, or just the way that I guess you perceive those things.
01:30:01
Speaker
And, and God, isn't that so interesting, too, to think that there's there are ways that we can impact
01:30:07
Speaker
the future growth of kids in a very positive way that isn't tied at all to what is getting measured on those assessments course and it is instead tied to the ways that we structure our classroom to value groups of learners
01:30:23
Speaker
And if we can, you know, put a value on those kids that are perhaps in struggling or underserved, you know, communities, then I wonder what, you know, the emphasis on putting the money into the things that are going to value those socio-emotional and those qualitative things could do.
Focus on Holistic Education Post-Pandemic
01:30:44
Speaker
instead of just going towards that test prep software.
01:30:47
Speaker
Again, right, there's the opportunity cost.
01:30:49
Speaker
There's that trade-off associated with what we do with those funds and the outcomes for these kids.
01:30:55
Speaker
Are we gonna use that money to pursue narrow test scores?
01:30:58
Speaker
Are we gonna use them to expand the child's experience of this return to school and deal with the trauma of the last 18 months?
01:31:08
Speaker
Exactly.
01:31:09
Speaker
I think it centers the idea.
01:31:10
Speaker
I mean, I guess,
01:31:12
Speaker
credit where credit's due, using the words unfinished learning instead of learning loss is a more positive framing of this.
01:31:21
Speaker
And it's going to affect how people view the concept.
01:31:25
Speaker
But there are better ways that we can view learners about the previous year that approach this from an even more positive attitude.
01:31:33
Speaker
And this isn't to be confused with like toxic positivity.
01:31:37
Speaker
We're not saying that like
01:31:39
Speaker
If you just treat all students super well that they're all going to be hyper successful.
01:31:45
Speaker
There's a lot more to it than just that.
01:31:48
Speaker
And there are students that need additional help that you're going to need to identify, like students, for example, on an IEP that are really struggling that need additional supports.
01:31:58
Speaker
That's fine.
01:32:00
Speaker
It's really ultimately just if we're approaching the school year saying, hey, we need to catch up and that's all we ever talk about, we shouldn't be surprised when our students struggle more.
01:32:10
Speaker
Instead of acknowledging the adversity and how they overcame it and how valiant they are and how they pushed through it and how they made this work and really celebrating the fact that school continued during a global pandemic, all these teachers banded together and made it work.
01:32:25
Speaker
And
01:32:26
Speaker
for the most part, even though it was quite messy originally, uh, I, I didn't see a lot of just complete chaos.
01:32:34
Speaker
Uh, people got through it, um, over time and there were a lot of positives.
01:32:38
Speaker
A lot of kids actually did better online.
01:32:40
Speaker
Uh, like they preferred socializing online.
01:32:42
Speaker
They liked the asynchronous elements.
01:32:44
Speaker
There are things that schools have adopted from this that they want to keep.
01:32:48
Speaker
Like some of them have developed like half days or more virtual days.
01:32:52
Speaker
They've established better PD for online learning to reach more learners.
01:32:56
Speaker
So instead of going, oh, my goodness, look at all this learning loss, we need to dial it back and go, quote unquote, back to basics, which we've repeated that ad nauseum for every single decade.
01:33:05
Speaker
We've gone back to basics, which is code for go back to hyper standardized, quote unquote, rigorous learning.
01:33:11
Speaker
Instead, we can make a school year that starts with hope and wonder and reimagination, reinvention.
01:33:19
Speaker
learning from what we just went through, reestablishing those connections and changing education for the better.
Future Educational Strategies
01:33:24
Speaker
It's not a lost narrative.
01:33:26
Speaker
It's something that we're actually gaining and changing and capturing.
01:33:30
Speaker
So if this is going to be the first part, right, where we're kind of talking about the what and the why part, in the next conversation that we'll have about this, we'll actually get to
01:33:43
Speaker
the latter half of the learning loss handbook, where we are actually going to address, like, how do we change that narrative?
01:33:50
Speaker
And how do we, you know, grab on to those assets that we do have?
01:33:55
Speaker
How do we recognize our losses?
01:33:58
Speaker
And how do we push forward
01:34:00
Speaker
to change education for the better.
01:34:02
Speaker
Because we have some tools and some templates and some activities that you can do both with yourself for like a personal reflection, you know, on what we've lost, what we've missed out on, what we'll do differently, but then how to bring those conversations in with your kids too.
01:34:19
Speaker
So again, like refocus on values and refocus on what kids missed out on,
01:34:27
Speaker
and how you can work to bridge that gap and address, again, the trauma that we've all been through of this last 20 months that has fallen disproportionately upon some people more than others for a variety of reasons.
01:34:43
Speaker
But thinking about how we can listen to kids, how can we actually start to address what is lost?
01:34:48
Speaker
How can we practice creative noncompliance?
01:34:52
Speaker
and push back against the inhumane structures that may be built up in place instead of this.
Listener Engagement & Support
01:34:58
Speaker
And then we'll kind of talk about some resources too that you can use to extend above and beyond all of this.
01:35:02
Speaker
So if you've made it this far, I mean, thank you so much for the time.
01:35:07
Speaker
It's not very often that we do sort of a loosely formatted thing like this, but if you like it, certainly let us know.
01:35:14
Speaker
This might be something we can do more of.
01:35:17
Speaker
If you hate it, on the other hand, you want a tighter, more scripted kind of thing, let us know that too.
01:35:22
Speaker
But thanks for sticking around.
01:35:24
Speaker
One more thing too.
01:35:26
Speaker
If you stayed along this long, we are in the middle of our funding drive.
01:35:29
Speaker
So if you are listening to this in September 2021,
01:35:33
Speaker
You could give to us right now, get a bunch of donor gifts.
01:35:37
Speaker
We're trying to reach a goal to continually produce these kind of resources because honestly, it's a lot of work.
01:35:43
Speaker
There are a lot of sleepless nights trying to look at graphs and charts.
01:35:47
Speaker
So if you want us to keep doing that, whether or not you like us and you think this is really interesting analysis, maybe you don't like us and want us to keep looking at graphs and data.
01:35:56
Speaker
That's up to you.
01:35:57
Speaker
But we really would appreciate your support.
01:35:59
Speaker
It goes a lot.
01:36:11
Speaker
Thank you again for listening to Human Restoration Project's podcast.
01:36:14
Speaker
I hope this conversation leaves you inspired and ready to push the progressive envelope of education.
01:36:19
Speaker
You can learn more about progressive education, support our cause, and stay tuned to this podcast and other updates on our website at humanrestorationproject.org.