It's become an annual ritual when U.S. News & World Report publishes its controversial “Best Colleges” rankings, leading to celebrations from the top and complaints from the bottom. Given the role that rankings play in the decisions of prospective students, most universities simply shrug and go along, using the rankings to their advantage as best they can.
But this year, universities should seriously consider their role in a process that misled students and their families.
All college rankings have their problems, but in recent years, U.S. News has adopted increasingly questionable methods for collecting and interpreting data, which has led some universities and colleges to drop out of the rankings process altogether. In a version released on September 18, U.S. News' rankings process has only gotten worse.
Most popular stories
Most Popular
This change, which U.S. News calls “the most significant methodological shift in the history of the rankings,” has caused some universities to soar in the rankings while others have dropped. Many universities have seen dramatic changes in their ranking positions. Vanderbilt University, which I oversee, moved from a tie for 13th place to a tie for 18th place, a fairly modest change. According to an analysis by our Institutional Research team, 107 universities (25% of those included in the “National Universities” ranking) have moved up 30 or more places.
Does this mean that the universities that dropped in the rankings are objectively worse than they were a year ago? Or that the universities that soared up the list are suddenly orders of magnitude better? Of course not. The fluctuations in the rankings are due primarily to changes in methodology. U.S. News' ever-changing methodology — what it counts and how much weight it gives it — is a subjective exercise that has little to do with the mission of our institution.
The goal of a research university like ours is to provide transformative education and groundbreaking research. To that end, we work hard to attract the most promising students and faculty and create a supportive university community where everyone can reach their full potential.
Providing opportunity for students of all backgrounds is a core value for our university and many other universities. For undergraduates, that means that we, like many other private universities, accept U.S. citizen (and qualified foreign) students regardless of their ability to pay, and fund 100% of each enrollee's demonstrated need without loans. Our generous financial aid allows all students to graduate from Vanderbilt without debt.
Last year, Vanderbilt University provided $366 million in financial aid, with $244 million of that going to undergraduates. Most of this undergraduate funding came through the university's Opportunity Vanderbilt student access program, which makes full undergraduate tuition available to nearly all families earning $150,000 or less. This means families can send their children to Vanderbilt for less than it costs in-state residents to attend many of the nation's top public universities.
Our efforts to increase access are paying off: Over the past decade, the number of our first-year students eligible for Pell Grants has increased by 70 percent, and the number of first-generation students has increased by more than 175 percent.
The U.S. News rankings ignore all of this because the data they use doesn't measure it. U.S. News emphasizes that its new methodology places more emphasis on social mobility, but it only uses students who receive federal aid to measure it, drawing data from its College Scorecard.
To see why this is misleading, consider two universities with 1,000 students each. University A has $12,000 in federal loans per student. Under U.S. News' current methodology, University A would report average student debt as $12,000, a metric that accounts for 5 percent of the overall ranking (up from 3 percent last year). But the average balance across all 1,000 students is actually just $12.
In contrast, University B has 1,000 students who each take out $10,000 in federal loans, meaning the average debt is $10,000, which is also what is reported in U.S. News. This gives the false impression that University B is less expensive.
Suppose University A wanted to “improve” its ranking. What should it do? Easy. It would simply encourage 999 students with zero debt to take on $100 each. Now each student would be included in US News' calculations, and the new metric would be ($12,000 + $99,900)/1,000 = $111.9. So, by increasing student debt, University A has skyrocketed in the rankings. Of course, encouraging students to take on more debt in order to improve their debt rankings is absurd and unethical. But these strange implications demonstrate the complete lack of validity of the US News student debt metric.
At Vanderbilt, 83% of students graduate without any debt. But when families choose to take on debt, it's usually to better manage their cash flow in the short term.
These misleading measures are important: Researchers have long pointed to the problem of “undermatching” — when high-achieving students from low-income or first-generation families don't apply to selective colleges even though they might be less expensive, severely limiting their educational opportunities.
The same methodological flaws apply to the rankings' career outcomes measure, a new addition to this year's rankings that uses College Scorecard data to track the percentage of graduates who earn more than the typical high school graduate four years after graduation.
But because College Scorecard tracks a limited group of students who received federal aid, the U.S. News rankings only considered about one-third of graduates. In other words, the U.S. News rankings don't place value on the career outcomes of more than two-thirds of students. This misconception is especially ironic considering the large amount of financial aid we offer is the main reason so few students rely on federal aid.
Additionally, this year's rankings removed important measures of academic quality, such as the percentage of faculty with the highest degrees in their fields, the percentage of enrolled students who were in the top 10 percent of their high school class, and average class size. Other factors that affect the quality of the student experience, such as the resources available to faculty, were given less weight.If a rankings scheme that ostensibly emphasizes academic quality is lowering the bar on academic quality, what exactly is it measuring?
These changes come after a year of turmoil at U.S. News & World Report. Last year, several law schools, led by Yale, withdrew from the rankings. There was similar turmoil in the medical school rankings, with some, including Harvard Medical School, dropping out. U.S. News is well aware of the universities' objections; our university and others have called out the rankings' flaws, but to no avail.
Some argue that colleges should ignore the annual rankings frenzy. But many students and families use rankings to help them choose a college, and that's where US News' latest rankings really fail. By undercutting academic quality and using biased data on tuition and career outcomes, US News has provided a misleading guide to help families, especially first-generation and low-income families, make one of the most important decisions of their lives.
The annual ritual of releasing the U.S. News rankings may not be over anytime soon, but colleges can no longer remain silent in good conscience.
Daniel Diermeier is president of Vanderbilt University.
