by JS O’Brien
Three-year-olds can be very trying, and not least because, once they find something that works for them, say, some action that made adults laugh, they’ll do it over and over and over and over expecting belly-bustin’ guffaws each time.
You’d think the venerable US News (formerly US News & World Report) would be too old for that sort of behavior, but it’s not. If the editors there can come up with some new ranking issue to “leverage the brand” they’ve built with their popular undergraduate college rankings, they’ll do it, and if they give a tinker’s dam if there is insufficient data to rank, or if their methodology is specious, they haven’t demonstrated it so far. Selling magazines is all, and the hell with those who get hurt.
US News’ most recent foray into the ranking business, their new raison d’etre, is the November 29, 2007 issue that is their first-ever ranking of US high schools. Their website asks themselves the question: “Why rank?” They answer their own question, saying “For accountability.” Great. Let’s have our high schools be accountable for doing their jobs well. I’m all for that. But I’m absolutely against measuring things that tell us almost nothing about whether high schools are doing their jobs well and pretending those measurements tell us something useful. And that is what US News has done.
The result is that, out of approximately 22,000 US high schools (their numbers), US News has picked roughly 1,600 to earn gold, silver, or bronze status, slightly more than 7% of all US high schools. Let’s take a look at three of the schools included in this elite 7%.
ThomasJeffersonHigh School for Science and Technology: TJ is a famous magnet school in Fairfax County, Virginia, one of the wealthiest and best-educated counties in the country. Admission is competitive. TJ takes only 20% of applicants based on admissions criteria very similar to those used in elite universities. The average SAT score at TJ is approximately 1470 (old scale), comparable to Harvard, Yale, and Princeton’s annual entering classes.
Peak to Peak Charter School: Peak to Peak is located in Boulder, Colorado, a town just named the “smartest city” in the US because of the number of people living there with advanced degrees, in another ridiculous ranking by Forbes Magazine. Anyone can get in to Peak to Peak in a lottery admissions process, but the school markets only to the most academically skilled children, so those without high academic skills almost always self-select out of the applicant pool. The school’s average ACT score is 25.1, which would equate to an average SAT of around 1130 to 1160.
Rural High: Rural High (not its real name because I have no intention of embarrassing a school that didn’t ask for this “honor”) is located in the rural South. About 40% of its students are economically disadvantaged. The community is far behind state averages in both high-school and college graduates. Almost half the students are African-American. The average SAT score is 904, more than 100 points below the national average, even though only about 30% of the students (presumably its top 30%) take the test. There is one AP class offered in calculus. Over the past five years, approximately 10 students have taken the AP calculus exam. Only one has passed it.
If you’re wondering what the hell Rural High has in common with the first two schools, the devil is in the methodology, isn’t it?
Simplified, US News uses only three criteria:
- The school outperforms other schools in its state on state-administered tests, adjusted for the school’s student demographics.
- African-Americans and Hispanics outperform their counterparts at other schools in the state.
- A school does well on a formula that compares the number of high school seniors who have ever taken an AP test to total seniors, and the number who got at least a “3” as a passing grade to total seniors.
In order to be in the “gold” category (only 100), a high school must jump the first two hurdles and then beat other high schools that jumped those hurdles on criterion 3. To be in the “silver” category (405 high schools), one must meet criteria 1 and 2, and get above a certain threshold on 3. To be a “bronze” school, one must meet only criteria 1 and 2.
This is why Rural High is one of the 7% recognized along with Thomas Jefferson and Peak to Peak. Rural High’s economically disadvantaged and minority students do very, very well compared to peer groups in their state, which earns the school bronze status, even though the college prep academic offerings at Rural High are anemic, to put it mildly.
With apologies to Dickens, the formula is an ass.
Let’s take Thomas Jefferson High. Passing the first two hurdles is a given, since no one gets into that school unless he or she is already a gifted student. Doubtless, TJ’s classrooms are rigorous and it is probably a superb place to prepare for college, but the idea that this school is doing a good job on criteria 1 and 2 is ludicrous. It’s not even relevant to this school.
How about Peak to Peak? Like TJ, it has a selective student body, though in this case, it’s self-selected. Peak to Peak’s ACT scores are very slightly better than those at a regular high school in the same district that has almost twice as many special ed and English language learners. Its self-selected students make passing the first two hurdles easy, and its relatively low ACT scores, given that self-selected student body, make me wonder why it’s not getting much better results on the ACT than it is.
Rural High does an outstanding job on criteria 1 and 2, or does it? The fact is, if students get to the 9thgrade or so unable to read, write, and/or do math at the high school level, there is precious little most high schools can do unless they are set up specifically for remedial instruction. If students are only slightly below grade level in reading and math, then a high school can, indeed, make a difference. The numbers on Rural High indicate that it is doing a great job of taking students who are already doing pretty well against their in-state peers (especially given the demographics) and making them better against those peers. Of course, it is the district’s elementary and middle schools that are doing the heavy lifting in the early years, but Rural High is still to be commended. But, let’s get real, here. How can a school that offers only one AP course, total, that has fewer than 1% of its students taking the AP test, and that has .1% of its students pass that test, at the lowest passing grade, be considered one of the US’s best high schools?
All this wouldn’t matter if these rankings didn’t lead to destructive parental and/or school district behavior. I got a phone call a few days ago from an educator at Rural High. They desperately need money to hire top teachers who can qualify to teach AP classes. They need better facilities (their lab facilities haven’t been upgraded since the mid-â€˜50s). The local city council has recently lost three bids to attract well-paying, white-collar businesses because the local schools can’t attract families from out of the area. But there will be no money forthcoming, because local voters say, “Why do you need more money? We already have one of the best high schools in America.”
I’m not one of those who attacks US News’ popular undergraduate college rankings. Given the data that are readily available, I think the magazine has found decent proxies for actual measurements of undergrad education quality which, to the best of my knowledge, do not yet exist. The issue here is decent proxies. Without direct evidence or decent proxy data, no ranking can be in the least useful and, in fact, may be quite harmful.
But, it sells magazines, doesn’t it? And if high school students at Rural High don’t get a shot at AP courses because they already attend one of the best high schools in America, well they can just go fish, can’t they? Why should anyone at US News care?