New Study on Charter School Applications Is Useful, in Measured Doses
How can we ensure charter school quality without recreating the bureaucracy that suffuses public education? That’s a persistent challenge. Last week, the Thomas B. Fordham Institute released a report analyzing factors in charter school applications that predict mediocre performance in a school’s first two years of operation. Used wisely, the report offers some real value. But as with so much education research, how it’s used matters immensely.
The analysis, done by Basis Policy Research’s Anna Nicotera and David Stuit, defined poor performance as the school scoring below the 25th percentile in student proficiency and below the 50th percentile in student growth on state reading and math tests. Studying 542 applications in four states, the researchers flagged three factors linked to mediocre early test scores: not having named a school leader or hired a management organization; providing little direct support for at-risk students; and using a child-centered curriculum, such as Montessori or Waldorf. The researchers calculated that these factors substantially boosted the probability of poor performance.
Such research has clear potential to be useful, especially for charter school authorizers. In 2015, for instance, an AEI report found that many authorizers aren’t all that sure about how to predict school success. As a result, many use the principle of “better safe than sorry” when deciding what to ask for in the application. This yields massive, overstuffed applications that, the AEI analysis suggested, could be trimmed by one-third without sacrificing authorizers’ ability to evaluate quality. Meanwhile, just last year, Whitney Bross and Douglas Harris of the Education Research Alliance of New Orleans analyzed factors in charter applications that might predict future performance—and came up empty. So this study has obvious value.
At the same time, this is the kind of education research that should be read with particular caution and care—but too rarely is. A couple of cautions are especially worth keeping in mind.
For starters, note just how narrow the researchers’ definition of school performance is. Reading and math performance may not provide a fair assessment of how a school that focuses on the arts or the sciences or caters to struggling students is performing, especially in that school’s first two years. It’s probably no great surprise that the results peg Montessori schools as “risky,” given that these schools don’t emphasize performance on state tests. The question is whether this really means that charter authorizers should regard student-centered models with skepticism.
Also, despite the authors’ assertion to the contrary, some of these measures are highly susceptible to gaming. For instance, it makes sense that schools which don’t name a leader in their application might be at risk of underperforming because the lack of leadership can reflect disorganization in the proposed school. However, as soon as naming a school leader becomes part of a scoring rubric, applicants have every incentive to name one. (The report suggests that gaming this metric would be difficult because there aren’t enough “good” leaders, but an applicant doesn’t need to name a “good” leader to check this box—any minimally qualified person will do.) In other words, this is one of those signals that loses value almost as soon as it is discovered.
However, the report does a pretty good job of acknowledging these limitations. For instance, the authors note, “Our intent is not to stifle innovation in the charter sector by suggesting that authorizers deny every application with one or more of these risk factors.” Rather, they contend that charter authorizers should use the report “to determine which charter applicants merit more thorough review” or may require “additional support.”
The risk, like with any report in this era of social media and short attention spans, is that observers read the title—“Three Signs that a Proposed Charter School Is at Risk of Failing”– and skip over all those caveats. That would be bad science, create bad policy and advance a troubling vision of charter school quality. It would also be a missed opportunity to ask hard questions about charter school authorizing and to get smarter about how to nurture an array of diverse, high-quality schools.
Frederick M. Hess is director of education policy studies at the American Enterprise Institute and author of the new book Letters to a Young Education Reformer. Jenn Hatfield is a research associate at AEI.