Despite heavy criticism, the Shanghai Report attracts always more media attention

Published 2021-07-23 07:21:34
Sky photo created by -

The Shanghai Ranking Consultancy of China releases an annual report on the Academic Ranking of World Universities (ARWU). It is considered by some to be a trustworthy and recognised list of university rankings in the world, but it has also raised hackles and criticism from many academics for bias and inaccurate indicators.

This report is usually released in August every year and rates universities on various indicators, like academic and research performance. About 2000 universities are actually ranked but only the top 1000 are published.

Criteria featured in the Academic Rankings of global universities

We list below the criteria used by the Shanghai report (ARWU) to classify universities.

  1. Quality of education: the total number of alumni who have been awarded Nobel Prizes or Field Medals (awarded for Mathematics every four years to candidates under the age of 40) from the university being ranked. These alumni could have achieved a bachelor's degree, master’s or doctoral qualifications from that academic institution. The year in which the degree was awarded also influences the score given.
  2. Awards: the number of academic personnel who have been awarded a Nobel Prize in the specific areas of chemistry, physics, economics and medicine; they also consider Field Medals in mathematics. The time of the award and the proportion of the award (if the award is given to more than one person) is also taken into account and that weighs the scoring differently.
  3. Highly Cited: the number of times researchers from that institution were cited in academic publications.
  4. Nature & Science: the number of articles published in the journal Nature & Science. Different scores are attached depending on how involved the author was in the published paper.
  5. Publications: the number of articles that appear in the Science Citation Index-Expanded (SCIE), which is double weighted, and Social Science Citation Index (SSCI) the year before.
  6. PCP: the total score of the above criteria is divided by the number of permanent academic staff.

Top rankings for 2020

  1. Harvard University achieved the highest score of 100
  2. Stanford University
  3. University of Cambridge
  4. Massachusetts Institute of Technology (MIT)

Click here for the full list.

The French University of Paris-Saclay (a research-intensive academic campus) ranked at number 14 (1st in the world for Mathematics, 9th in the world for Physics-1st in Europe and 25th for Medicine and Agriculture), the first non-US or non-UK university from the top of the list. Famed King's College London made the list at number 47.

Academic Ranking of World Universities logo

The list is made up of largely-known universities, having achieved the top positions in the rankings previously. However, 3 Chinese newcomers entered the list in 2020 for excelling in their subjects: Southeast University (Transportation Science & Technology), Xi`an Jiaotong University (Mechanical Engineering), and Xidian University (Telecommunication Engineering).

Top universities in Natural Sciences (Mathematics, Physics, Geography, ...etc):

  • Paris-Saclay
  • University of Oxford
  • MIT
  • University of California, Berkeley

Top rankings in Engineering:

  • MIT
  • Harvard
  • Southeast University
  • Xi`an Jiaotong University

Top Life Sciences Rankings:

  • Harvard
  • Wageningen University and Research (Netherlands)
  • Ghent (Belgium)

Medical Sciences’ Top Universities:

  • Harvard
  • University of Michigen-Ann Arbor
  • University of Pennsylvania

Top positions for Social Sciences:

  • Harvard
  • Yale
  • University of Amsterdam
  • Erasmus University Rotterdam

The Shanghai Ranking Consultancy has already released the 2021 list of Academic Subjects which rank different academic institutions based on the subjects they teach. Some of the subjects they consider are the natural sciences, engineering, social sciences, medical sciences, etc.

The subject rankings also use similar criteria as the university rankings like: research quality, international collaboration, academic achievements and awards.

How reliable is the Shanghai rankings?

The accuracy and authenticity of this type of ranking is especially important because they garner a lot of media attention, and are viewed by many students and their families as a useful tool when choosing an academic institution. Results can also impact political decisions like making reforms in higher education. The listed institutions also use this status to advertise their exalted standing among other institutions, which can be very misleading. Potential employers may also look at this list and favour alumni from only certain institutions.

Despite the heavy criticism from other academics since the very first ranking in 2003, every year a new list is published with even more media attention, and some countries even view it as the gold standard to achieve global prominence.

Initially, the authors admitted to devising the ranking as a way to understand how to close the academic gap between Chinese universities and international ones. They were unable to procure data from international institutions so decided to look at only research and academic performance as markers of university achievement.

Problems with the criteria used

  1. Nobel Prizes are usually awarded long after the actual research was done. This lag period distorts the ranking of the university, i.e this factor reflects more of the past qualities (when the research was actually done) of an academic institution rather than its present research potential, and is therefore an inaccurate measure.
  2. By looking at this criterion for awards and prizes it places more value on historical reputation and prestige which results in the top positions being dominated by the oldest and largest universities.
  3. There are also other prestigious awards like the A.M Turing Award for Computer Science that are not considered at all. Therefore, looking at only Nobel Prizes and Field Medals is inaccurate and does not cover all other important scientific areas.
  4. On the subject of highly cited researchers, the heavy reliance on a single authorising body like the Thomson Scientific citation counts skews the data because it favours mostly areas like medicine and biology, and due to the long period over which it records (20 years), most of those researchers cited are older and may have changed institutions multiple times in their careers, therefore giving the incorrect scoring for the current academic institution.
  5. The authors of the Shanghai Rankings do not make their raw data available for independent scrutiny which makes it impossible to fully confirm their results. Critics have clearly seen that all the criteria used in compiling the list are only loosely based on what the authors intended to depict. They only made use of data they were able to easily acquire whilst ignoring other important parameters.
  6. Another basic, and very important consideration ignored in the ranking is which institutions comply with the definition of "university" - the UK and US definitions may differ from those of mainland Europe for instance (in France, a dual system with Business Schools and Engineering Schools attracts a large part of the "elite" while most Universities concentrate on more academic subjects). Currently, the Shanghai ranking promote a biais toward the anglo-saxon type of higher education.

These are only some of the highlighted criticisms of the Shanghai Rankings. There are many more inaccuracies and inconsistencies which may explain why US and UK have multiple academic institutions that trust all the top positions whilst other countries do not feature high in the list (France) or not at all within the first 500 positions (India).

Alternative rankings

Critics of ARWU claim that the ranking criteria only looks at academic and research, which is a very narrow perspective and therefore only gives a very limited view of an academic institution.

It may be more informative to look at alternative rankings that illustrate more diverse indicators and your personal selection criteria better, like the QS World University Rankings and Times Higher Education World University Ranking (THE) university rankings. Less well-known rankings like the French ENSMP uses the criterion of "the number of alumni from a particular academic institution that have become CEO of a company listed in the Fortune Magazine" and provides a complete different ranking.

There is no perfect method of determining which is the best university. Students can look at factors such as: student satisfaction data, employment opportunities, scholarships, how competitive it is to get admission and how the programme ranks domestically rather than globally - this is often a better indication of the success of the institution.

Share your experience, participate in the discussion and leave comments in our forum HERE.

Facebook Twitter LinkedIn Pinterest WhatsApp Addthis


Author: KashGo
Expat Mum in the Desert and content writer for

For other discussions, advice, question, point of view, get together, etc...: please use the forum.

More articles

- My Life Abroad -
A selection of expat stories

"A fun compulsive read!"
J. Matcham, Amazon

"I strongly advise people ready to live abroad to read this book!"
Patrice, Amazon