More and more professors are coming forward to express their skepticism about university ranking systems. This time they are talking about the ranking of business programs.
Bloomberg Businessweek, which publishes an annual MBA ranking, has for years misrepresented submitted data, according to Anjani Jain, associate dean of the Yale School of Management. Jain says he’s not the only one with concerns and other business school deans and professors have doubts about the reliability of widely read MBA rankings.
In February, math professor Michael Thaddeus revealed faulty data provided by Columbia University to US News and World ReportThis is the ranking of the best universities. The revelation caused Columbia, ranked no. 2 in 2021, falling to no. 18 spot. Meanwhile, the former dean of Temple University’s business school was convicted of wire fraud last year, after he submitted false data that ranked the university’s online MBA program #1 by American News from 2015 to 2018.
MBA rankings have a profound impact on business school applications and program size. The top 10 programs have expanded, with more applicants, money and faculty, because of the rankings, according to Kenneth Brown, a professor at the University of Iowa’s Tippie College of Business. “There are very real consequences for these rankings, and they are probably more dramatic for business schools – professors have always been concerned that it doesn’t really help students, but it has become a reality. that we have to deal with,” Brown said.
Jain from Yale, who considers himself an admirer of Thaddeus, first raised issues with Businessweek’s MBA Ranking in 2021. “I sort of accidentally discovered a problem last year,” he said.
Factors such as “compensation” carry disproportionate weight
business week brings together data from about 84 different US business schools on five different components: compensation, learning, networking, entrepreneurship, and diversity. “The unique feature of this ranking is that they don’t impose their own view on the degree of evaluation of each of these factors,” Jain said. business week instead asks respondents what the weights should be, using stakeholder-generated weights for each component. However, Jain ran into trouble when reviewing the public data released by business week alongside its 2021 ranking. When deducing what each component’s weights should be to replicate the ranking, it found different results from the stakeholder-generated weights. “You can’t replicate the ranking order they released using the data they released and the methodology they described,” he said.
With Businessweek’s 2022 rankings, Jain found the same problem, explaining the data distortion in an article by Poets & Quants, a website that focuses on business schools. Compensation, given a 37% weighting generated by stakeholders, was given a 69% weighting by business week, while the weight given to learning dropped from 25% to 6%. “It’s dishonest. They keep claiming the virtue of stakeholder-generated weights, while their calculations significantly skew the weights,” Jain said.
Jain believes that applying the stakeholder-generated weightings correctly would create a very different ranking – although the top five schools would remain more or less unchanged, the rest of the list would be significantly revamped. However, he said business week probably doesn’t want to change its methodology, because a very different listing could raise questions about the credibility of its past rankings. “It reminds me of the Chinese saying, ‘when you ride a tiger, you can’t easily get off a horse,'” Jain said.
In a statement provided to the Observer, a Bloomberg Businessweek The spokesperson said the publisher stands by its ranking.
Beyond business week, Jain believes that MBA rankings in general are flawed, reducing multi-faceted student experiences to mere linear orders. “But if you want to undertake this, at least do it with some integrity,” he said, adding that business week is not the only MBA ranking with problems.
Self-reported data and peer reviews raise questions of validity
Challenges have already been raised about the validity of data provided to ranking publishers, said Homer Erekson, a professor at Texas Christian University’s Neely School of Business. Erekson believes that aspects of peer review and self-reported data, common to all MBA rankings, are unreliable. And while business week and the FinancialTimesanother popular compiler of MBA rankings, publishes their methodologies, Erekson says that even those who take the time to verify rankings with methodologies are still relying on the validity of the data provided by the universities themselves.
The FinancialTimes did not respond to requests for comment.
Compensation, which measures post-graduation salaries, is often disproportionately prioritized in MBA rankings, Brown said. While an index like diversity typically doesn’t vary significantly from university to university, compensation differs significantly by region and industry, he said. And schools that place students in cities with high living costs, such as New York and San Francisco, will often reflect higher salaries. The variability of a factor like compensation will disproportionately underline its importance in the rankings.
“Where schools show the greatest differences will be what will be reflected in any kind of rankings,” Brown said. “They give credit to schools that place more people in higher paying industries and cities with higher costs of living. It’s hard to adapt to that. Although high salaries are a goal for many business students, Brown worries that the rankings place more emphasis on pay than other factors regarding the quality of education.
After surprising rankings, some publishers stopped their lists altogether
Recently, some leading MBA ranking publishers have discontinued their annual ranking lists. In August, Poets and quants reported Forbes would also discontinue its MBA ranking for 2022, citing the loss of an editor to compile the list. Meanwhile, The Economist in 2021 released a ranking that many business schools declined to participate in, citing data disruptions due to the Covid-19 pandemic. Fifteen of the top 25 schools skipped the 2021 rankings, resulting in a list that many criticized as spotty. The Economist in July announced that its 2022 MBA ranking would be its last.
The Economist and Forbes did not respond to requests for comment.
According to Brown, this type of backlash reveals how ranking editors are not open to change despite the problems inherent in their rankings. “If you end up with something that doesn’t reinforce people’s existing perceptions of status, people will question and not read your post,” he said. “But if so, doesn’t that just reinforce perceptions of permanent status?”