How US News is hurting our high school students

by JS O’Brien

Three-year-olds can be very trying, and not least because, once they find something that works for them, say, some action that made adults laugh, they’ll do it over and over and over and over expecting belly-bustin’ guffaws each time.

You’d think the venerable US News (formerly US News & World Report) would be too old for that sort of behavior, but it’s not. If the editors there can come up with some new ranking issue to “leverage the brand” they’ve built with their popular undergraduate college rankings, they’ll do it, and if they give a tinker’s dam if there is insufficient data to rank, or if their methodology is specious, they haven’t demonstrated it so far. Selling magazines is all, and the hell with those who get hurt.

Even children.

US News’ most recent foray into the ranking business, their new raison d’etre, is the November 29, 2007 issue that is their first-ever ranking of US high schools. Their website asks themselves the question: “Why rank?” They answer their own question, saying “For accountability.” Great. Let’s have our high schools be accountable for doing their jobs well. I’m all for that. But I’m absolutely against measuring things that tell us almost nothing about whether high schools are doing their jobs well and pretending those measurements tell us something useful. And that is what US News has done.

The result is that, out of approximately 22,000 US high schools (their numbers), US News has picked roughly 1,600 to earn gold, silver, or bronze status, slightly more than 7% of all US high schools. Let’s take a look at three of the schools included in this elite 7%.

ThomasJeffersonHigh School for Science and Technology: TJ is a famous magnet school in Fairfax County, Virginia, one of the wealthiest and best-educated counties in the country. Admission is competitive. TJ takes only 20% of applicants based on admissions criteria very similar to those used in elite universities. The average SAT score at TJ is approximately 1470 (old scale), comparable to Harvard, Yale, and Princeton’s annual entering classes.

Peak to Peak Charter School: Peak to Peak is located in Boulder, Colorado, a town just named the “smartest city” in the US because of the number of people living there with advanced degrees, in another ridiculous ranking by Forbes Magazine. Anyone can get in to Peak to Peak in a lottery admissions process, but the school markets only to the most academically skilled children, so those without high academic skills almost always self-select out of the applicant pool. The school’s average ACT score is 25.1, which would equate to an average SAT of around 1130 to 1160.

Rural High: Rural High (not its real name because I have no intention of embarrassing a school that didn’t ask for this “honor”) is located in the rural South. About 40% of its students are economically disadvantaged. The community is far behind state averages in both high-school and college graduates. Almost half the students are African-American. The average SAT score is 904, more than 100 points below the national average, even though only about 30% of the students (presumably its top 30%) take the test. There is one AP class offered in calculus. Over the past five years, approximately 10 students have taken the AP calculus exam. Only one has passed it.

If you’re wondering what the hell Rural High has in common with the first two schools, the devil is in the methodology, isn’t it?

Simplified, US News uses only three criteria:

  1. The school outperforms other schools in its state on state-administered tests, adjusted for the school’s student demographics.
  2. African-Americans and Hispanics outperform their counterparts at other schools in the state.
  3. A school does well on a formula that compares the number of high school seniors who have ever taken an AP test to total seniors, and the number who got at least a “3” as a passing grade to total seniors.

In order to be in the “gold” category (only 100), a high school must jump the first two hurdles and then beat other high schools that jumped those hurdles on criterion 3. To be in the “silver” category (405 high schools), one must meet criteria 1 and 2, and get above a certain threshold on 3. To be a “bronze” school, one must meet only criteria 1 and 2.

This is why Rural High is one of the 7% recognized along with Thomas Jefferson and Peak to Peak. Rural High’s economically disadvantaged and minority students do very, very well compared to peer groups in their state, which earns the school bronze status, even though the college prep academic offerings at Rural High are anemic, to put it mildly.

With apologies to Dickens, the formula is an ass.

Let’s take Thomas Jefferson High. Passing the first two hurdles is a given, since no one gets into that school unless he or she is already a gifted student. Doubtless, TJ’s classrooms are rigorous and it is probably a superb place to prepare for college, but the idea that this school is doing a good job on criteria 1 and 2 is ludicrous. It’s not even relevant to this school.

How about Peak to Peak? Like TJ, it has a selective student body, though in this case, it’s self-selected. Peak to Peak’s ACT scores are very slightly better than those at a regular high school in the same district that has almost twice as many special ed and English language learners. Its self-selected students make passing the first two hurdles easy, and its relatively low ACT scores, given that self-selected student body, make me wonder why it’s not getting much better results on the ACT than it is.

Rural High does an outstanding job on criteria 1 and 2, or does it? The fact is, if students get to the 9thgrade or so unable to read, write, and/or do math at the high school level, there is precious little most high schools can do unless they are set up specifically for remedial instruction. If students are only slightly below grade level in reading and math, then a high school can, indeed, make a difference. The numbers on Rural High indicate that it is doing a great job of taking students who are already doing pretty well against their in-state peers (especially given the demographics) and making them better against those peers. Of course, it is the district’s elementary and middle schools that are doing the heavy lifting in the early years, but Rural High is still to be commended. But, let’s get real, here. How can a school that offers only one AP course, total, that has fewer than 1% of its students taking the AP test, and that has .1% of its students pass that test, at the lowest passing grade, be considered one of the US’s best high schools?

All this wouldn’t matter if these rankings didn’t lead to destructive parental and/or school district behavior. I got a phone call a few days ago from an educator at Rural High. They desperately need money to hire top teachers who can qualify to teach AP classes. They need better facilities (their lab facilities haven’t been upgraded since the mid-‘50s). The local city council has recently lost three bids to attract well-paying, white-collar businesses because the local schools can’t attract families from out of the area. But there will be no money forthcoming, because local voters say, “Why do you need more money? We already have one of the best high schools in America.”

I’m not one of those who attacks US News’ popular undergraduate college rankings. Given the data that are readily available, I think the magazine has found decent proxies for actual measurements of undergrad education quality which, to the best of my knowledge, do not yet exist. The issue here is decent proxies. Without direct evidence or decent proxy data, no ranking can be in the least useful and, in fact, may be quite harmful.

But, it sells magazines, doesn’t it? And if high school students at Rural High don’t get a shot at AP courses because they already attend one of the best high schools in America, well they can just go fish, can’t they? Why should anyone at US News care?

9 replies »

  1. From the Dean of the School of Medicine where I work:

    “No one likes a popularity contest, but everyone likes encouragement. No one believes that the US News and other polls are scientific or reliable, unless they have us in the top ten.”

    i.e., we hate it, but we have to play along or we look bad.

  2. Mmm. But you’ve missed a very important difference. US News’ medical school rankings are defensible based on the criteria they use. The high school rankings are indefensible.

  3. We disagree on that point, JS – I don’t think any of these rankings are defensible in any way.

    After all, the schools recognized as great are the usual suspects – Princeton, Yale, Harvard, Stanford, et. al. – and everyone else is just looking to get into a category where their marketing depts. (which exist by siphoning off funds from actual educational pursuits) can use the info to “sell” education like it was freakin’ fast food. I’m reminded of my late brother-in-law who was always getting me to play in golf tournaments. I was a 6 handicapper; he was a 26. Yet the way “flights” (i.e., categories of golfers) were put together for competitive purposes, he invariably won trophies and I was an also ran competing against scratch golfers.

    What this will do is take that model – which now dominates college education – and apply it at the high school level – and so kids at “Rural High” will get cheated even more than they’re being cheated now so that the district superintendent’s office can hire marketers to “sell” high schools the same way colleges get sold. And we move another step closer to privatizing education instead of making the civic commitment to creating a well educated citizenry.

    You know, as I think about that, that notion of civic commitment to an educated citizenry sounds like it ought to be part of the Bill of Rights. The new 10A Amendment, as it were….

  4. Jim:

    Yes, we’ll have to disagree. I plan to publish a defense of US News’ undergrad rankings when they next come out (next April, I believe, but I could be off a few months). But, just as an appetizer, let me put forth one factor the publication uses in its undergrad rankings: quality of undergraduate class as measured by standardized test scores and rank in class.

    Now, I’ve had many, many conversations with your colleagues who have actually taught at schools where the entering class was no better than average on those two measures. They tell me that they had to dumb down the curriculum to match the students’ abilities, and that As are given out on an implied curve (that is, the best of an average bunch gets an A regardless of the level of perfection of the wor). I have also had conversations with faculty at elite schools who tell me that no dumbing down is necessary in their classes.

    So, wouldn’t the quality of the student body be a reasonable proxy datum for at least one dimension of quality of education? I mean, all other things being equal, wouldn’t it follow that faculty at Harvard, Yale, and Princeton can present material at a level that faculty at, say, North Southwestern Clayton County State College can’t?

  5. It’s a marketing game, isn’t it – for both USN and the schools that do well in its rankings (at the college level – I don’t think it’s doing the marketing dept at Rural HS any good).

    Honestly, I’m pretty sure that if I had the time and inclination I could pull together a rip-roaring indictment of its college rankings, too, but the HS methodology is simply horrific.

    But what does its marketing group care, really?

  6. Doc Slammy,

    Well, if you think of “marketing game” as including one of the four Ps, “product,” then I would make the argument that all US News has done with its college issue is to find a crying need for information and then fill that need with the best product (rankings) currently available. But marketing also includes brand leveraging (as you know better than I), and one good product can be used as credibility to sell a number of bad products.

    And I think this is what’s going on with the high school ranking issue.

  7. My son ended up at Pineview School for the gifted in Florida (#6 I think) from second grade through eighth. He liked it there and we were happy with the faculty. However, my son wanted to go to four years of Phillips Exeter, which isn’t found in most conventional rankings. Although pricey, Exeter was the best investment I ever made in my son. After Exeter, he ended up at Yale College, which is another good investment. Although those schools have turned him into an unabashed liberal, they did teach him how to think, and I’m good with that.


  8. Jeff:

    Philips Exeter doesn’t publish stats, therefore, it doesn’t get into rankings. Actually, I’m not sure any private schools do.

  9. JS

    My high school published stats in the 70’s, and soon dropped the publication because their stats were going downhill. Hell, they let me graduate….