'Rankings regime': New college rankings are out, but should we trust them?

BIRMINGHAM, Ala. (WIAT) — U.S. News and World Report has published its latest college and university rankings, but should we trust them?

On Monday, the outlet released the 37th edition of its “best colleges” rankings. U.S. News touts itself as the “global authority in education rankings,” but how much should the public actually rely on these statistics? What do they actually measure, and what do they miss?

Dr. Cecilia Orphan, associate professor of higher education at the University of Denver, is an expert on the rankings and said that if she had a magic wand, she’d do away with them altogether.

To begin, Orphan said that the origin of the “best college” rankings is important context for understanding what they are and the value they have.

“It was actually at a time when the the magazine itself was facing real significant financial trouble,” Orphan said. “And so the rankings were a way — a new business offering in the magazine and actually save the magazine from going under… It wasn’t necessarily to evaluate colleges and universities. It was part of a a magazine with a profit motive that was looking for another thing to rank or place consumer value on.”

Now, she said, the rankings have become a “machine in higher education” that can have an outsized impact on the choices student make about college.

One reason this can be a problem is because the numbers used in producing the rankings may include problematic data and exclude other, more important or useful information.

The data used to create the rankings “often tells us very little about the educational outcomes or the learning or the experience that a college can provide,” Orphan said.

For example, many of the statistics used to create the rankings involve “input statistics” like test scores, numbers that may reflect the beginning of a student’s college experience, but don’t capture anything after, and may also disadvantage certain types of students.

“When we’re looking at what are the incoming test scores of students,” Orphan said, “it’s really problematic because we know that the best predictor of ‘success’ or a high score on the ACT or the SAT is how wealthy your family is.”

Sometimes, the numbers are misrepresented by universities themselves, who have an incentive to report the “best” statistics possible.

Orphan also points to the fact that a reputation survey accounts for 20% of the score that results in a college’s ranking as an issue.

“So this is a survey that gets sent out to deans and presidents and provosts around the country, and they are without probably ever stepping on a college campus, without having the time or resources to really look and evaluate their institutions, expected to rank them,” she said.

However, Orphan said there is important data the outlet’s methodology excludes, too. For example, that numbers on diversity, job placements, and success after college aren’t included in the rankings.

U.S. News and World Report’s role in what Orphan termed “the rankings regime” is also an issue because of the amount of power it gives the outlet in influencing decisions about university administration, she said.

“You have presidents of colleges and universities that are evaluated on whether or not they’re able to increase the institution’s rank or to improve it” Orphan said. “And it’s just baffling to me that this for-profit magazine has that much control over higher education. You know, how many policymakers would like to have that much control over higher education?”

So if the rankings aren’t the best source of information for prospective students, where should they turn? Orphan said there are other sources of information that student can rely on, among them the U.S. Department of Education’s College Scorecard. Another source of good information to inform college decisions: current students.

“When I’m interviewing prospective students for our our master’s or our doctoral programs,” she said, “I always tell them: here’s five students, reach out to them and they’ll tell you what their experience has been like.”

Visiting a campus yourself can be helpful, too, and since the beginning of the pandemic, virtual tours have become more common.

But Orphan said she knows there’s a need for easily digestable, quick information that can help inform college decisions, something U.S. News and World Reports knows well.

“U.S. News and World Report has really capitalized on the fact that a lot of people, either they don’t have the time or they don’t really have the interest to go that deep,” she said. “And so they’re going to trust something like newspapers and World Report, which they think saves them time.”

This comes with a risk, though, she said:

“It’s an open question as to whether or not what they how they think about quality through that magazine aligns with how they themselves would think about quality as a student.”

Below are the 2022 U.S. News and World Report rankings for Alabama’s colleges and universities, which appear in four separate categories. You can also see their college scorecards by searching here.

National universities

School	Ranking
Auburn University	99
Samford University	136
University of Alabama (tie)	148
University of Alabama Birmingham (tie)	148
University of Alabama Huntsville	263
University of South Alabama	299 to 391