View More View Less
  • 1 Universidad de Vigo, ETSE TELECOMUNICACION, Campus Universitario, 36310, Vigo, Spain
Restricted access

Abstract

We take a new look at the Shanghai Jiao Tong Academic Ranking of World Universities to evaluate the performance of whole university systems. We deal with system aggregates by means of averaging scores taken over a number of institutions from each higher education system according to the Gross Domestic Product of its country. We treat the set of indicators (measures) at the country level as a scale, and investigate its reliability and dimensionality using appropriate statistical tools. After a Principal Component Analysis is performed, a clear picture emerges: at the aggregate level ARWU seems to be a very reliable one-dimensional scale, with a first component that explains more than 72% of the variance of the sample under analysis. The percentages of variance of the indicators explained by the first component do shed light on the fact that ARWU is in fact measuring the research quality (both at the individual and collective levels) of a university system. When the second principal component is taken into account, the two principal components contribute to explain more than 90% of the variance. The rotated solution facilitates the interpretation of the components and provides clear and interesting clustering information about the 32 higher education systems under analysis.

  • Bartlett, M. S. 1954 A note on the multiplying factors for various chi-square approximations. Journal of the Royal Statistical Society 16 B 29698.

    • Search Google Scholar
    • Export Citation
  • Billaut, J. C., Bouyssou, D., Vincke, P. 2010 Should you believe in the Shangai ranking: An MCDM view. Scientometrics 84 1 237263 .

  • Brown, T. A. 2006 Confirmatory factor analysis for applied research 3 The Guilford Press New York.

  • Catell, R. B. 1966 The scree test for number of factors. Multivariate Behavioural Research 1:245276 .

  • Dehon, C., McCathie, A., Verardi, V. 2010 Uncovering excellence in academic rankings: A closer look at the Shanghai ranking. Scientometrics 83 2 515524 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • DeVellis, R. 2003 Scale development: Theory and applications 2 Sage Thousand Oaks, CA.

  • Docampo, D. (2008). Rankings internacionales y calidad institucional. Revista de Educación, Número Extraordinario, 149176.

  • Florian, R. V. 2007 Irreproducibility of the results of the Shangai academic ranking of world universities. Scientometrics 72 1 2532 .

  • Guadagnoli, E., Vellicer, W. 1988 Relation of sample size to the stability of component patterns. Psychological Bulletin 103:265275 .

  • Hsu, J. 1996 Multiple comparisons: Theory and methods Chapman Hall London.

  • IMF. (2009). World Economic Outlook (WEO) Database: Downloaded from the International Monetary Fund server on November 19th 2009. http://www.imf.org/external/pubs/ft/weo/2009/02/index.htm.

    • Search Google Scholar
    • Export Citation
  • Jackson, J. E. 2003 A users guide to principal components Wiley Hoboken, New Jersey.

  • Kaiser, H. 1974 An index of factorial simplicity. Psychometrika 39:3136 .

  • Liu, N. C., Cheng, Y. 2005 Academic ranking of world universities: Methodologies and problems. Higher Education in Europe 30 2 127136 .

  • Marginson, S. (2005). There must be some way out of here. Tertiary Educ. Management Conference, Keynote address, Perth, Australia.

  • Morrison, D. 2000 Multivariate statistical methods 3 McGraw-Hill New York.

  • Stevens, J. (1996). Applied multivariate statistics for the social sciences. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.

  • Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson Education, Inc./Allyn and Bacon.

  • Zitt, M., & Filliatreau, G. (2007). The world class universities and ranking: Aiming beyond status (pp. 141160), Romania: UNESCO-CEPES, Cluj University Press, chap Big is (made) beautiful: Some comments about the Shangai ranking of world-class universities, Part Two.

    • Search Google Scholar
    • Export Citation

Manuscript submission: http://www.editorialmanager.com/scim/

  • Impact Factor (2019): 2.867
  • Scimago Journal Rank (2019): 1.210
  • SJR Hirsch-Index (2019): 106
  • SJR Quartile Score (2019): Q1 Computer Science Apllications
  • SJR Quartile Score (2019): Q1 Library and Information Sciences
  • SJR Quartile Score (2019): Q1 Social Sciences (miscellaneous)
  • Impact Factor (2018): 2.770
  • Scimago Journal Rank (2018): 1.113
  • SJR Hirsch-Index (2018): 95
  • SJR Quartile Score (2018): Q1 Library and Information Sciences
  • SJR Quartile Score (2018): Q1 Social Sciences (miscellaneous)

For subscription options, please visit the website of Springer

Scientometrics
Language English
Size B5
Year of
Foundation
1978
Volumes
per Year
4
Issues
per Year
12
Founder Akadémiai Kiadó
Founder's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Publisher Akadémiai Kiadó
Springer Nature Switzerland AG
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
CH-6330 Cham, Switzerland Gewerbestrasse 11.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 0138-9130 (Print)
ISSN 1588-2861 (Online)