Why the U.S. News Best High School Rankings Are Flawed
Last week, U.S. News & World Report released its 2017 Best High Schools rankings. The top three spots, and five of the top ten, went to Arizona charter schools from the same provider – BASIS Schools Inc. At first glance, that’s pretty impressive, but first impressions are often deceiving. While U.S. News does its best to provide an evenhanded ranking system, two related problems keep it from capably shouldering the burden.
First, U.S. News does not have the authority to determine which high schools are “best.” Undaunted by that lack of authority, it has created a ranking system whose standard is the product of the limits to the available data. That system lists the public schools with the highest Advanced Placement passing rates on top and ultimately offers an impoverished view of what makes certain high schools the “best.”
Quibbling over the U.S. News rankings isn’t new and isn’t all that helpful in isolation. But it’s worth doing because the authority the rankings claim to have deserves checking, and because the process of checking them shines a light on how states, which do have the authority and responsibility to rate schools, can improve the way they go about it.
Each year, U.S. News teams up with RTI International to run 20,000 public high schools through a four-step process to rank which are the best. In step one, they evaluate schools’ proficiency rates on state math and reading tests against statistical expectations given their student poverty rates. Passing schools move to step two, in which U.S. News assesses whether historically disadvantaged students performed better than the state average. In step three, U.S. News cuts all schools whose graduation rate is below 75 percent (somewhat odd, given that the national average is 83 percent). In step four, schools are ranked on a “College Readiness Index,” which is based entirely on their success in Advanced Placement courses.
What makes a school "best" in the U.S. News rating system? A school's broader performance on state tests has to be moderately above average to clear the first three steps, but that left more than 29 percent of the schools moving on to step four this year. After that, it all comes down to AP passage rates. BASIS schools unapologetically push AP participation, which is why they dominated the top ten rankings. No doubt, AP success is a high bar for high school students, and since the AP tests are the same nationwide, it provides a usable metric for academic excellence. But is it a good enough indicator to decide which high schools are best?
The answer is no. The reason U.S. News leans so heavily on AP is that the data are available. But that is like the proverbial drunk looking for his keys underneath the street lamp. The rankings promote the notion that the best high schools are the ones with the highest outcomes, and because AP success is the only outcome measure they have, they use it, even if the way the top schools generate those outcomes is dubious practice.
For instance, the College Board, which runs the AP program, advises that AP courses are not generally appropriate for ninth grade students. However, BASIS schools and Dallas’s School for the Talented and Gifted maximize AP by expecting ninth graders to take those courses. These schools have impressive results, and the pressure they place on students may work within that school, but it certainly does not mean those schools are the most productive.
The problem with looking under the street lamp is that the rankings primarily gauge where students end up, not where they start from or how much they learn. The BASIS schools dominating the top ten push advanced academics hard and are transparent about the fact that the workload is not a fit for all students. Other schools in the top ten have GPA requirements for enrollment. It’s good that there are hard-charging schools for advanced students, but it’s irresponsible to ignore how selective they are. In focusing narrowly on AP outcomes, U.S. News leaves the impression that all schools have equivalent starting points when, in reality, it’s nearly impossible for non-selective schools to end up at the top of this list.
In fairness, U.S. News is arguably doing the best it can with the available data. Data needed to gauge student learning growth are not available in ways that could be applied to all schools. And the rankings do incorporate some measures of student disadvantage, although these only apply weakly in the first two steps. The problem is that their work is branded as ranking which schools are best, but their methods don’t back that up. From my perspective, high schools whose students make the most progress are the ones that should be deemed the best and imitated, not the ones that use selectivity to get the best outcomes.
States have more discretion over how they rate schools, and their ratings have more practical implications for schools and parents. Fortunately, a window of opportunity is open for states to improve how they do that. Since 2001’s No Child Left Behind Act, states have used the percentage of proficient students as their primary measure of school success. But proficiency does not gauge student growth and is all too often determined by students’ achievement level coming into the school. However, under the Every Student Succeeds Act, states now have the flexibility to use measures of growth rather than proficiency to gauge school quality. So far, six of the 18 states that have filed ESSA plans with the U.S. Department of Education have elected to do so.
States have a much greater responsibility for rating schools than U.S. News does. State school ratings have clear consequences for schools, and states have the data systems and authority to decide what information should be brought to bear in their ratings. As the remaining states develop their ESSA plans, they should include measures of growth that show which schools are helping students make the most progress. Without these and other more useful measures of school quality, the public will be left in the dark, looking for answers under street lamps.
Nat Malkus is a research fellow in education policy studies at the American Enterprise Institute.