Does the Media Cover School Choice Fairly? Yes and No.
On both sides of the school choice debates, there’s a longstanding conviction that media coverage of new research findings is systematically unfair. Former USA Today editorial writer Richard Whitmire sounded a familiar refrain when he lamented that “a stunning report” on the success of New Orleans charter schools had been roundly ignored. “The national press,” he wrote, “maintained total radio silence.” Given prior evidence that the mainstream media plays favorites in reporting on education policy, these concerns of media bias are worth taking seriously.
Since news coverage plays an important role in shaping popular opinion and public policy, it’s worth asking if reporting on research and evidence is indeed tilted. Fortunately, during the past decade, a number of accomplished researchers have rigorously examined school choice outcomes. Because the results of such studies vary — even when they examine the same locale or are conducted by the same scholars — we can observe whether similar studies draw diverging coverage based on which side of the debate they seem to support.
In other words, when reporting on otherwise similar studies, does the media pay more attention to those that are more or less positive about school choice outcomes? To find out, we identified five pairs of studies. The two studies in each pair measured the same outcomes, used the same basic methodology, and were either conducted by the same researchers or else conducted in the same locale. The only significant difference in each pair was the research result.
It’s worth saying a bit about the five pairs of studies.
In 2009, Stanford University’s CREDO used longitudinal student-level data to analyze charter schools nationwide. CREDO found that nearly half of charter schools performed no differently than traditional public options when it came to reading and math, but that over a third delivered learning results that were significantly worse. In 2013, CREDO repeated the study and found very different outcomes, with charter students posting reading gains substantially larger than those of students in district schools and math results that were significantly improved from 2009. The first study was regarded as a blow to charter schools, the second as quite positive for them.
In 2008, University of Arkansas professor Patrick Wolf and several colleagues evaluated Washington D.C.’s Opportunity Scholarship Program via a randomized controlled trial design. At the two year mark, they found no statistically significant achievement effects for students who were offered vouchers, but their third year results in 2009 found that voucher recipients gained three months of learning in reading compared to the control group. The first study was seen as showing few benefits from vouchers, the second as suggesting substantial benefits from them.
Northwestern University’s David Figlio examined the Florida and Ohio school choice programs using distinct but broadly similar methodologies. In 2014, Figlio produced the final of a series of reports on Florida’s tax credit scholarship program. He found no significant changes in reading or math outcomes for scholarship recipients. In 2016, Figlio and Krzysztof Karbownik analyzed the Ohio voucher program by comparing students who received vouchers to similar public school students. He found that voucher recipients who attended private schools did worse on reading and math than their counterparts in traditional public schools. The Ohio study was regarded as a negative finding for voucher programs, the Florida study as neutral for them.
In 2016, Duke University’s Atila Abdulkadiroglu and colleagues studied the Louisiana Scholarship Program one year after implementation. They found that voucher recipients had lower academic achievement, performing significantly worse in math, reading, science, and social studies. In 2017, the University of Arkansas’ Jonathan Mills and Patrick Wolf used the same methodology to re-examine the program at the three-year mark, and found that the earlier losses had essentially vanished. The first study was regarded as highly negative for school vouchers, the second as modestly positive — given that it suggested progress as well as a reversal of the earlier negative results.
Finally, in 2015, Carnegie Mellon’s Dennis Epple and colleagues conducted an international literature review (including research that used methods other than randomized control trials). They found vouchers had no consistent effects on student achievement. In 2016, the University of Arkansas’s Danish Shakeel, along with some colleagues, analyzed the evidence from 19 international randomized controlled trials of school vouchers, finding that vouchers tended to raise both reading and math scores.
We tallied the number of major media news stories and editorials that mentioned each of these studies. We searched one international source (The Economist) and five national sources (The New York Times, The Wall Street Journal, USA Today, The Washington Post, and the Los Angeles Times). In each case, keyword searches included combinations of the researcher(s), the school choice program, and the location studied. To ensure consistency, we utilized the search engine LexisNexis, a database that collects news articles from national and international news outlets.
What did we find? It turns out that news stories in these outlets played no favorites when covering school choice research. Thirteen news stories cited a “positive” study and 15 referenced a “negative” study. While some kinds of studies received much more attention than others, coverage was relatively balanced within the five pairs. The coverage was also relatively balanced for each of the various outlets. For example, The New York Times published two articles referencing a positive study and three articles referencing a negative study; for The Washington Post, the figures were five and three.
While the news coverage was quite balanced, the editorial pages were another story. Editorials and op-eds in these newspapers mentioned negative school choice studies twice as often as they did positive studies, with 36 mentions of negative studies compared to 18 of positive studies. Newspaper editorials featured 18 mentions of negative studies versus only eight mentions of positive studies, and about the same ratio was evident across columns and op-eds.
So, major media news coverage appears to be largely impartial when reporting on research in the contested field of school choice. At the same time, the editorial staffs at the most influential newspapers have clearly opted (as is their right) to tout negative research findings much more frequently than positive ones. Given that editorials reference school choice research findings at a substantially higher rate than does news coverage, it’s easy to see why some observers may regard media coverage as tilted, even when news coverage itself is not. When the nation’s most influential newsrooms play favorites, they should be called out. We’ve done that before. But it’s also fair to give due credit when reporters play it straight, as they have when it comes to school choice research.
Frederick M. Hess is director of education policy studies at the American Enterprise Institute. RJ Martin and Brendan Bell are research assistants at AEI.
