printThe Impact of Changes in the World Ranking Methodologies on University Positions
Trostyanskaya I.B.
Cand. Sci. (Polit.), Director of the Directorate of Research and Analysis, State Autonomous Sociological Research Center, Moscow, Russia. trostyanskaya@sociocenter.info
Polikhina N.A.
Deputy Director, State Autonomous Sociological Research Center, Moscow, Russia. polihina@sociocenter.info
Trostyanskaya I.B., Polikhina N.A. The Impact of Changes in the World Ranking Methodologies on University Positions. Sotsiologicheskie issledovaniya [Sociological Studies]. 2018. No 9. P. 71-79
The article considers the main changes in the methodologies of world’s renowned rankings, their impact on university positions and comparability of ranking results over time. The QS and THE rankings’ methodological changes come under special scrutiny. The ARWU ranking is regarded as having an unchanged methodology. The subject of the research consists of universities entering the top 300 groups in the QS, THE and ARWU rankings, as they form an extended pool of leading universities in the international market. The article analyzes how the changes in methodology impact the positions of universities in the top 100, top 200 and top 300 groups from 2015 till 2017, as well as their relative positioning in the above-mentioned groups measured by correlation coefficients. Besides, the authors study how many universities maintain their positions in the top 100, top 200 and top 300 groups over two consecutive years (2015/2016 and 2016/2017), as well as their correlation coefficient. The research shows that the changes in the world ranking methodologies either do not impact, or exert a minor influence on university positions in the top 300 group of the above mentioned rankings. All the rankings under consideration demonstrate a high degree of continuity. These findings confirm that the ranking agencies take a careful approach towards methodological changes in order to ensure data comparability across time and enable the advancement of universities in the rankings (provided that universities make deliberate efforts to increase their competitiveness).
Aguillo Isidro F., Bar-Ilan Judit, Levene Mark, Ortega Jose’ Luis. (2010) Comparing university rankings. Scientometrics. 85: 243–256. DOI 10.1007/s11192-010-0190-z
Balatsky E. V., Ekimova N. A. (2011) The Comparative Reliability of the Global University Rankings. The Journal of the New Economic Association. No. 11: 127–140. (In Russ.)
Balatsky E. V., Ekimova N. A. (2011) The State University of Management, Moscow The International Rankings of Universities: Practice of drawing up and using. The Journal of the New Economic Association. No. 9: 150–172. (In Russ.)
Bar-Ilan Judit, Levene Mark, Lin Ayelet (2007) Some measures for comparing citation databases. Journal of Informetrics. No. 1: 26–34.
Fagin Ronald, Kumar Ravi, Sivakumar D. (2003) Comparing top k lists. SIAM Journal on Discrete Mathematics. No. 17(1): 134–160.
Kehm B. M. (2016) Global University Rankings – Impacts and Unintended Side Effects. Sotciologicheskie Issledovaniya [Sociological Studies]. No. 8: 71–80. (In Russ.)
Kincharova A. (2013) World University Rankings: methodology, effects and critics. URL: http://past-centre.ru/wp-content/ uploads/2013/04/Metodologiya_mirovyh_reitingov_universitetov.pdf (accessed 16.02.2018) (In Russ.)
Neudachin I. G. (2017) Comparison of World University Rankings. Actual Problems of Humanities and Natural Science. No. 9(1): 5–8. (In Russ.)
Polikhina N. A., Trostyanskaya I. B. (2018) University Rankings: Trends, Methodology, Changes/The Ministry of Education and Science of the Russian Federation. Moscow: State autonomous Sociological Research Center. (In Russ.)
Salmi Jamil, Alenoush Saroyan (2007) League tables as policy instruments: Uses and misuses. In: Higher Education Management and Policy: 31–68.
Salmi Jamil (2009) The Challenge of Establishing the World Class Universities. Moscow: Ves’ Mir. (In Russ.)