How College Rankings Actually Work

Ever wonder how a particular school’s ranking can be wildly different from one set of rankings to another? It’s Randy Stiles’ job as Grinnell’s associate vice president for analytic support and institutional research to know exactly how that happens. He explains:

“What these ranking systems all do in one way or another,” Stiles says, “is put together some collection of schools — liberal arts colleges, research universities, publics, privates — and rank them all on one long list. Then there is some collection of measures that are given some collection of weights. All of that gets added up into an overall score, from which is produced an ordered list.”

What makes one set of rankings more influential than another depends to a large extent on commercial reach. “Readership matters a lot,” Stiles says. “Some of these things have a lot of readership, and people give greater credibility to them. Others, not so much.”

Making sense of college rankings would be easier if the all of the rankings systems produced similar results. In many cases they don’t, and Grinnell is a perfect case in point. Grinnell was No. 18 among national liberal arts colleges in the 2018 U.S. News rankings.

In the other systems that Stiles tracks, the College came in at No. 19 on Washington Monthly, but at Nos. 73, 9, 156, 31, and 54 on other lists.

“U.S. News puts a big emphasis on reputation and resources,” Stiles explains. “Forbes claims to emphasize outcomes or ‘output over input.’ Kiplinger’s is about best value. Money, not surprisingly, is about affordability and the salaries of graduates. The New York Times Access Index emphasizes the percentage of Pell students and the economic diversity. College Factual is outcomes-focused.

“These days, there is more and more talk about outcomes,” Stiles continues, “and when people say outcomes in these systems they’re talking about graduation rates and salaries more than anything else.” That’s an important distinction, he says, because rankings that weigh earnings heavily can skew data unfavorably for graduate students whose higher incomes materialize on a longer timeline.

Also, not all systems are equal in terms of their own development. Stiles says Forbes’ ranking of Grinnell since 2008 has fluctuated by “an incredible variation” of 80 points.

“I can guarantee you Grinnell didn’t change that much between 2008 and 2012, but the system changed, as did the way people were using it and the way it was managed,” Stiles says. “So it’s important to remember that the systems themselves need time to mature and achieve stability.”