COVID-19 Has Left Millions Of Students Behind. Now What? – FiveThirtyEight
March 4, 2022
If a kid isnt keeping up with peers academically, summer school seems like a no-brainer. Instead of forgetting what they learned during the school year while theyre on vacation for two months, theyre catching up and getting ahead. Which is why it was a surprise when a Rand Corporation study of summer school programs in five urban school districts found that this common-sense solution didnt really solve the problem.
Rands study found that summer school offered modest, short-term improvements in math scores at best, but those improvements faded by the fall. Other metrics performance in language arts, student attendance and overall grades showed no meaningful link to summer school. The effects were pretty underwhelming, said Megan Kuhfeld, a senior research scientist with NWEA, a nonprofit educational testing and research organization.
Overall, summer school programs didnt deliver on their promises. But some subgroups did benefit: the students who regularly attended the programs that were better at navigating hurdles like student retention.
Its perhaps never been so urgent to make educational interventions like summer school work for kids. Two years into the pandemic, children across the nation are behind where they would have been academically if the pandemic hadnt happened. To help bridge the gap, educational theories will have to adapt to the unique realities of actual kids lives and families needs. If they dont, even the best ideas, with tons of evidence behind them, wont work in the real world.
Kids learned plenty during the pandemic, Kuhfeld told me. The problem, she said, is that they arent learning as much or as quickly as they were each year before the pandemic. Nationally, third-graders in fall 2021 were, on average, testing significantly below where third-graders were testing in fall 2019 in reading and math. The NWEA assessments showed these declines extended across third-graders through eighth-graders, too.
Most of the experts I spoke to said the popular term learning loss is a misnomer its not that kids have lost ground, theyre just not progressing as fast. But the slower progression is real, and there are patterns to it. The effects were particularly pronounced among Black, Hispanic and American Indian and Alaska Native students.
In the NWEA data, the median percentile ranks for Black third-graders went down 10 points in reading and 14 points in math. For white third-graders, the median percentile ranks declined by exactly half of that (5 points in reading and 7 in math), while the median percentile ranks for Asian American third-graders fell by 3 points in both subjects.
In addition, theres evidence of declines in attendance and high school graduation rates, something that could signal a broad sense of emotional disconnection from school. Which, in turn, could help explain slowed learning or exacerbate it, said Dan Goldhaber, director of the National Center for Analysis of Longitudinal Data in Education Research.
Slowed learning during the pandemic doesnt necessarily mean kids are doomed, however. In fact, other researchers like Torrey Trust, a professor of learning technology at the University of Massachusetts Amherst, said kids actually learned a lot of things during the pandemic that they might not have learned otherwise. For many, virtual classes meant more time with family, more skills with technology, and for some, even better educational experiences, free from bullying.
The other good news: Research shows that the slower progress documented by these test scores should be able to be fixed with small-group tutoring. Its not rocket science, said Thurston Domina, a professor of educational policy and organizational leadership at the University of North Carolina at Chapel Hill. You get the kids in small groups and you can really give them customized instruction and focus on them. When Matthew Kraft, a professor of education and economics at Brown University, reviewed several meta-analyses of the effectiveness of various educational interventions in 2021, he found that tutoring in small groups had a significantly greater effect on student test scores than changes in class size, longer school days or summer-school-type programs.
But while its relatively simple for researchers to run studies on classrooms or schools and figure out which interventions produce the best results, its hard for educators to take those findings and put them to work across America. The evidence doesnt produce a solution it just shows you how hard its going to be to craft a wide-reaching solution.
Case in point: those summer school studies. One of the biggest factors affecting the overall failure of summer school programs in the Rand analysis was that only around half the kids who attended one year didnt come back for the next and some kids didnt even attend each day the first year. The kids who attended summer school habitually, for both years, did improve their math and language skills in ways that lasted all school year. But that group represented only about 35 percent of all the kids involved in the study.
So summer school works just fine if you can get kids to actually go. And that sets up a whole other set of logistical complications that have to be studied and analyzed and implemented. It takes hiring the right teachers who have the motivation and specific interest in teaching summer school, Kuhfeld said. It also takes long-term dedicated recruitment of kids into the programs. Unlike with regular school, students dont have to attend summer school, so getting them and their families to choose the programs means you have to build both interest and trust neither of which is a given. And all of this takes money. Theres a big gap between what should work in theory and what works in practice, Kuhfeld said.
This kind of effect is depressingly common. When the George W. Bush administration set up a program to compile evidence-based educational resources in 2002, education specialists told me theyd hoped this program the What Works Clearinghouse would bridge the gap between academia and classrooms. They envisioned it as a way for teachers to get a better handle on how to use evidence-based interventions in the classroom. We thought we would punch in third-grade math and get an answer, said Rachael Gabriel, a professor of literacy education at the University of Connecticut.
But it never worked out to be that simple.
In many cases, researchers I spoke to found that teachers the people tasked with educating students and bringing those test scores up didnt have much control over which interventions they could use and how. Those decisions were made higher up in the chain of administration. A teacher might want to try something and not be allowed. Or they might be excited to try something that was allowed but not be given the funding or staff or bus transport to make it happen effectively.
Making things work in a classroom is different from making things work in a whole district or a whole state or the whole country. Thats something Domina learned when Californias State Board of Education tried to mandate all eighth-graders to take and be tested on algebra. The idea was very much based on evidence, he said. Studies showed that separating some kids into elite math and others into remedial math served to widen inequality and narrow kids futures. Giving kids higher expectations leads them to do better. So expanding access to algebra for all should have reduced test-score gaps between rich and poor, white and Black.
But it didnt. In fact, the opposite happened. Domina sees problems of scale particularly staffing issues at the heart of that failure. Offering algebra to everyone meant that schools needed a lot more algebra teachers, and quickly. But there were only so many fully qualified, highly skilled algebra teachers. A lot of kids, particularly the ones in lower-income schools, ended up with teachers who didnt have as much experience and werent as effective at teaching the material, he said.
That story is particularly poignant now. Small-group tutoring can help students catch up on what they didnt get a chance to learn during the pandemic. But small-group tutoring takes staff and schools are one of many industries suffering from staffing shortages. Experts like Kraft are concerned that schools might create failing tutoring programs by using irregular volunteers or older students in place of dedicated staff.
Much like students, schools themselves arent necessarily functioning at a neutral, pre-pandemic state, either. The biggest trend Ive seen in the last 6-12 months is that schools are struggling to get the basics down. Staying open is hard, said Chase Nordengren, the principal research lead for Effective Instructional Strategies at NWEA. Hes seen many cases where federal funds, which otherwise may have been spent on staffing tutoring programs to mitigate learning loss, were spent instead on things like better ventilation, personal protective equipment and substitute teachers.
I think tutoring is a really promising initiative, Goldhaber said. But we have never tried to do tutoring at the scale that we are trying it today. Because of that, he said, parents should be advocating for real-time evaluation and course-correction to go along with these learning-loss interventions. There should be tools in place to help teachers know when something isnt working for their specific school and allow them to make the kind of personalized adjustments we know are necessary to make any intervention effective. But that, again, takes resources.
In the end, its not kids pandemic test scores that really make researchers feel gloomy about the future of education. Instead, its the way educational systems have been set up to fail those kids. Schools have been running with limited resources and little wiggle room for change for at least the past decade, Domina said. And now weve hit a crisis. And theyre not resilient.
Read more:
COVID-19 Has Left Millions Of Students Behind. Now What? - FiveThirtyEight
