Scientometrics
Volume/Issue:
Volume 86: Issue 1
On using the Shanghai ranking to assess the research performance of university systems
Author: Domingo Docampo 1
View More View Less
  • 1 Universidad de Vigo, ETSE TELECOMUNICACION, Campus Universitario, 36310, Vigo, Spain
Pages:
77–92
Online Publication Date:
29 Aug 2010
Publication Date:
01 Jan 2011
Article Category:
Research Article
DOI:
https://doi.org/10.1007/s11192-010-0280-y
Keywords:
University system; Ranking; Scale; Reliability; PCA; Clustering; Shanghai
Restricted access

Abstract

We take a new look at the Shanghai Jiao Tong Academic Ranking of World Universities to evaluate the performance of whole university systems. We deal with system aggregates by means of averaging scores taken over a number of institutions from each higher education system according to the Gross Domestic Product of its country. We treat the set of indicators (measures) at the country level as a scale, and investigate its reliability and dimensionality using appropriate statistical tools. After a Principal Component Analysis is performed, a clear picture emerges: at the aggregate level ARWU seems to be a very reliable one-dimensional scale, with a first component that explains more than 72% of the variance of the sample under analysis. The percentages of variance of the indicators explained by the first component do shed light on the fact that ARWU is in fact measuring the research quality (both at the individual and collective levels) of a university system. When the second principal component is taken into account, the two principal components contribute to explain more than 90% of the variance. The rotated solution facilitates the interpretation of the components and provides clear and interesting clustering information about the 32 higher education systems under analysis.

  • Bartlett, M. S. 1954 A note on the multiplying factors for various chi-square approximations. Journal of the Royal Statistical Society 16 B 296–98.

    • Search Google Scholar
    • Export Citation

  • Billaut, J. C., Bouyssou, D., Vincke, P. 2010 Should you believe in the Shangai ranking: An MCDM view. Scientometrics 84 1 237–263 .

  • Brown, T. A. 2006 Confirmatory factor analysis for applied research 3 The Guilford Press New York.

  • Catell, R. B. 1966 The scree test for number of factors. Multivariate Behavioural Research 1:245–276 .

  • Dehon, C., McCathie, A., Verardi, V. 2010 Uncovering excellence in academic rankings: A closer look at the Shanghai ranking. Scientometrics 83 2 515–524 .

    • Crossref
    • Search Google Scholar
    • Export Citation

  • DeVellis, R. 2003 Scale development: Theory and applications 2 Sage Thousand Oaks, CA.

  • Docampo, D. (2008). Rankings internacionales y calidad institucional. Revista de Educación, Número Extraordinario, 149–176.

  • Florian, R. V. 2007 Irreproducibility of the results of the Shangai academic ranking of world universities. Scientometrics 72 1 25–32 .

  • Guadagnoli, E., Vellicer, W. 1988 Relation of sample size to the stability of component patterns. Psychological Bulletin 103:265–275 .

  • Hsu, J. 1996 Multiple comparisons: Theory and methods Chapman Hall London.

  • IMF. (2009). World Economic Outlook (WEO) Database: Downloaded from the International Monetary Fund server on November 19th 2009. http://www.imf.org/external/pubs/ft/weo/2009/02/index.htm.

    • Search Google Scholar
    • Export Citation

  • Jackson, J. E. 2003 A users guide to principal components Wiley Hoboken, New Jersey.

  • Kaiser, H. 1974 An index of factorial simplicity. Psychometrika 39:31–36 .

  • Liu, N. C., Cheng, Y. 2005 Academic ranking of world universities: Methodologies and problems. Higher Education in Europe 30 2 127–136 .

  • Marginson, S. (2005). There must be some way out of here. Tertiary Educ. Management Conference, Keynote address, Perth, Australia.

  • Morrison, D. 2000 Multivariate statistical methods 3 McGraw-Hill New York.

  • Stevens, J. (1996). Applied multivariate statistics for the social sciences. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.

  • Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson Education, Inc./Allyn and Bacon.

  • Zitt, M., & Filliatreau, G. (2007). The world class universities and ranking: Aiming beyond status (pp. 141–160), Romania: UNESCO-CEPES, Cluj University Press, chap Big is (made) beautiful: Some comments about the Shangai ranking of world-class universities, Part Two.

    • Search Google Scholar
    • Export Citation
Cover Scientometrics
Scientometrics
Print ISSN:
0138-9130
Online ISSN:
1588-2861

To see the editorial board, please visit the website of Springer Nature.

Manuscript submission: http://www.editorialmanager.com/scim/

For subscription options, please visit the website of Springer Nature.

Scientometrics
Language English
Size B5
Year of
Foundation		 1978
Volumes
per Year		 1
Issues
per Year		 12
Founder Akadémiai Kiadó
Founder's
Address		 H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Publisher Akadémiai Kiadó
Springer Nature Switzerland AG
Publisher's
Address		 H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
CH-6330 Cham, Switzerland Gewerbestrasse 11.
Responsible
Publisher		 Chief Executive Officer, Akadémiai Kiadó
ISSN 0138-9130 (Print)
ISSN 1588-2861 (Online)

Monthly Content Usage

Abstract Views Full Text Views PDF Downloads
Feb 2022 9 0 0
Mar 2022 6 1 1
Apr 2022 7 1 0
May 2022 1 1 2
Jun 2022 13 3 3
Jul 2022 6 0 0
Aug 2022 0 2 4