Studies show covid’s toll on students living in poverty, learning from home

Most school districts saw declines, but the magnitude varied.

Those are the findings from more than a half-dozen studies published in recent months examining the pandemic’s toll on academic achievement. Across-the-board, they find big drops between spring 2019, before the pandemic hit, and spring 2021, one year in.

“The pandemic was like a band of tornadoes, leaving devastating learning losses in some districts and leaving many other districts untouched,” said Tom Kane, faculty director for the Center for Education Policy Research at Harvard University.

Students made more progress last year, but it was nowhere near enough to make up for the losses already sustained.

“People were hoping, ‘Oh gosh, there’s going to be a lot of natural bounce back that occurs,’ and we did not see it last year,” Kane said. “Maybe it will happen this year, but I’m not sure there’s much evidence underlying that hope.”

The high price of distance learning

One of the fiercest debates during the pandemic’s first year was how quickly schools should reopen and how significant the ramifications would be of keeping them closed. We now have some answers.

A pile of evidence charts setbacks that were more severe the longer students stayed in virtual school. These studies examined the impact of in-person vs. remote education during the 2020-21 school year, when policies varied widely. In Texas and Florida, Republican governors ordered schools to operate in person starting in fall 2020. Elsewhere, and often in big cities, resistance and fear of the virus among teachers and parents kept schools virtual for a year or longer.

Different studies rely on different data sets and describe the magnitude of the impact to varying degrees, but they all point in the same direction:

· A study using data from the testing company NWEA found modest academic declines for students who quickly returned to in-person classes in fall 2020. But achievement losses were far higher for those who learned from home, and they were most pronounced for students in high-poverty, mostly remote schools, widening long-standing racial and economic achievement gaps.

Students who were in person full-time during 2020-21 lost an average of 7.7 weeks of learning in math. But those who were in virtual class for more than half the year lost more than double that — an average of 19.8 weeks.

This research was based on NWEA assessments of 2.1 million students in 10,000 districts and analyzed by researchers at NWEA, Harvard and the American Institutes for Research.

· An Ohio study found that reading achievement in school districts that went fully remote fell, on average, two or three times as much as it did for those studying in person during the 2020-21 school year.

It looked closely at third-graders, because these students take reading tests in the fall and spring, so growth over the course of a school year can be assessed. During the 2020-21 school year, those who learned remotely fell twice as far behind as those in person, compared with what would be expected in a pre-pandemic year.

“The more weeks of remote learning, the less students learned during that time-period,” said Vladimir Kogan, a political scientist at Ohio State University, who produced these reports.

For math, the relationship in the Ohio data was less clear, with drops most severe for students whose districts employed a mixture, or hybrid, of in-person and remote learning.

· A study of state test scores in 11 states by Brown economist Emily Oster and others found districts with full in-person learning saw smaller declines than those that operated remotely, with hybrid systems in-between. This research, based in part on data Oster collected during the pandemic, also found in-person school was more common in districts that had higher test scores to start with and that had fewer Black and Hispanic students.

· A project called the Education Recovery Scorecard, a collaboration between researchers at Harvard and Stanford universities, looked at test results from school districts in 29 states. It found that the average fully remote district lost more academic progress than others in the same state that operated in person, particularly for math but also for reading.

Using this data, Nat Malkus, an education researcher at the American Enterprise Institute, divided school districts into three “buckets” based on how much their students were remote or in person. He calculated that students in the most remote grouping lost 60 percent of a school year in math, while those who spent the most time in classrooms lost 44 percent of a year.

For reading, the most remote group lost 33 percent of a year, vs. 19 percent of a year for the most in-person group.

“There clearly is an association between the duration of remote instruction and students’ learning loss,” he said. But he added: “It’s also not as clean a relationship as everyone expected.”

That’s because there was tremendous variation across the country, with scores in both remote and in-person districts ranging widely. And there was a major outlier: California, where schools took a long time to return but academic achievement was not particularly bad relative to other states.

Sean Reardon, director of the Educational Opportunity Project at Stanford and a project leader on the covid analysis, said Malkus’s calculations looked correct, but emphasized that remote or in-person learning explained only part of the variation.

His team is working to see what other factors might account for the rest of the differences, such as local coronavirus rates or economic conditions. He speculated that parents’ financial woes, illness and social isolation all played a part.

“To reduce the educational impacts of the pandemic to whether or not learning happened remote or in person is to miss all the other ways the pandemic has disrupted kids and parents and teachers’ lives,” he said. “There is a relationship but it’s not the only thing.”

High poverty, steep declines

Not surprisingly, the students who were already facing the biggest challenges suffered the biggest setbacks.

The Education Recovery data shows that students in the school districts with the highest poverty rates lost the equivalent of two-thirds-of-a-grade in math, compared with the lowest-poverty districts, who lost just under half-a-grade. The same was true for reading, though the gap was smaller. High-poverty districts lost 31 percent of a grade, vs. 25 percent in low-poverty schools.

The analysis of NWEA data found that high-poverty schools were more likely to go remote in the first place, and when they did so, they suffered larger declines than the low-poverty schools that did the same.

The report found 30 percent of the difference in achievement losses in math between high- and low-poverty schools could be attributed to the increased likelihood that high-poverty schools were remote, and 50 percent was due to the impact of learning virtually.

“Remote instruction was a primary driver of widening achievement gaps,” the report found.

Several studies show that students are crawling out of the holes they fell into, though not every student and not as quickly as needed to reach the academic growth expected pre-pandemic.

A national study using 2022 NWEA data found in the case of younger students, the learning last year was close to pre-pandemic levels, helping students begin to catch up. But given the steep declines of the previous year, students were still far behind, particularly in high-poverty schools.

The research also found the rebound stronger in math than in reading, which is important given that math took a bigger hit to start with.

Also encouraging: Renaissance, another testing company, found that last year, students grew academically at about the rate that would be expected in a pre-pandemic year.

But again, some subgroups of students grew at faster rates than would be expected, including Asian American, Pacific Islander and White students. Hispanic and particularly Black students grew more slowly than expected, as did students with disabilities.

“What alarms me the most are the widened inequities we’ve seen,” said Karyn Lewis, director of the Center for School and Student Progress at NWEA. “Everyone’s been harmed but some have been harmed more than most.”

Bigger kids, bigger problems

Several studies show that older students are not recovering as quickly as younger ones. This trend is masked by much of the research, because many of the state tests are administered only through eighth grade. But others include older students.

The Ohio data, for instance, showed that students in grades three, four and six made up at least half of the lost ground in reading. Seventh-graders made up some ground, though not as much. There was scant improvement in eighth grade, and in grade 10, scores dropped again.

In math, there was modest progress in most grades, but in 10th, there was virtually none.

That worries Kogan, the Ohio State researcher who did the analysis. “You’re talking about high school students with just a few years left,” he said. “We don’t have that much time left to get them back on track. … The older students should be our top priority.”

The NWEA research from 2022 also found that younger students were catching up much faster than older students.

The Renaissance data, which includes every grade, showed the same. For reading, growth was about as expected or higher last year for students in grades five and younger, but lower than expected for all those older than that.

The same pattern held for math, with students in grades nine and up seeing slower than typical growth in the 2021-22 school year.

For these kids, the downward spiral continues, said Gene Kerns, vice president and chief academic officer.

“The recovery is actually playing out in very different ways for different kids,” he said. “The kids in our elementary schools have weathered this much better. It seems the older the kid, the more lingering the impacts.”