View More View Less
  • 1 Statistics Department, Universidad Carlos III de Madrid, C/Madrid 126, Getafe 28903, Spain
Restricted access

Abstract

Composite indicators play an essential role for benchmarking higher education institutions. One of the main sources of uncertainty building composite indicators and, undoubtedly, the most debated problem in building composite indicators is the weighting schemes (assigning weights to the simple indicators or subindicators) together with the aggregation schemes (final composite indicator formula). Except the ideal situation where weights are provided by the theory, there clearly is a need for improving quality assessment of the final rank linked with a fixed vector of weights. We propose to use simulation techniques to generate random perturbations around any initial vector of weights to obtain robust and reliable ranks allowing to rank universities in a range bracket. The proposed methodology is general enough to be applied no matter the weighting scheme used for the composite indicator. The immediate benefit achieved is a reduction of the uncertainty associated with the assessment of a specific rank which is not representative of the real performance of the university, and an improvement of the quality assessment of composite indicators used to rank. To illustrate the proposed methodology we rank the French and the German universities involved in their respective 2008 Excellence Initiatives.

  • Aguillo, I, Bar-Ilan, J, Levene, M, Ortega, JL 2010 Comparing university rankings. Scientometrics 85:243256 .

  • Bastedo, M., & Bowman, N. (2010). College rankings as an interorganizational dependency: Establishing the foundation for strategic and institutional accounts. Research in Higher Education. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Centre for Higher Education Development (CHE), UNESCO European Centre for Higher Education (CEPES), Institute for Higher Education Policy (IHEP). (2006). Berlin principles on ranking of higher education institutions, Berlin. Available online at: http://www.che.de/downloads/Berlin/Principles/IREG/TEXTsymbol/534.pdf.

    • Search Google Scholar
    • Export Citation
  • Chen, Y, Liu, NC 2008 Examining major rankings according to the Berlin principles. Higher Education in Europe 33:201208 .

  • Cook, JM 1958 Rational formulae for the production of a spherically symmetric probability distribution. Mathematical Tables and Other Aids to Computation 11:8182.

    • Search Google Scholar
    • Export Citation
  • Debreu, G 1960 Topological methods in cardinal utility theory KJ Arrow S Karlin P Suppes eds. Mathematical methods in the social sciences Stanford University Press Stanford 1626.

    • Search Google Scholar
    • Export Citation
  • Dill, D, Soo, M 2004 Is there a global definition of academic quality? A cross-national analysis of university ranking system. Public Policy for Academic Quality University of North Carolina Chapel Hill, NC.

    • Search Google Scholar
    • Export Citation
  • Ding, J., & Qiu, J. (2011). An approach to improve the indicator weights of scientific and technological competitiveness evaluation of Chinese universities. Scientometrics, 86, 285297. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Federkail, G 2008 Rankings and quality assurance in higher education. Higher Education in Europe 33:219231 .

  • Geraci, M., & Degli, M. (2011). Where do Italian universities stand? An in-depth statistical analysis of national and international rankings. Scientometrics. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harvey, L. (ed.) (2008). Rankings of higher education institutions: A critical review. Quality in Higher Education, 14 (3), 187207.

  • Marsaglia, G 1972 Choosing a point from the surface of a sphere. The Annals of Mathematical Statistics 43:645646 .

  • Pike, GR 2004 Measuring quality: A comparison of U.S. News Rankings and NSSE Benchmarks. Research in Higher Education 45 2 193208 .

  • Sadlak, J, Merisotis, J, Liu, NC 2008 University rankings: Seeking prestige. Raising visibility and embedding quality—the Editors’ views. Higher Education in Europe 33:195199 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Saisana, M., & D’Hombres, B. (2008). Higher education rankings: Robustness and critical assessment. JRC Scientific and Technical Reports, European Commission.

    • Search Google Scholar
    • Export Citation
  • Saisana, M, Saltelli, A, Tarantola, S 2005 Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. Journal of the Royal Statistical Association A 168:307323 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stolz, I, Hendel, D, Horn, A 2010 Ranking of rankings: Benchmarking twenty-five higher education ranking systems in Europe. Higher Education 60:507528 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Usher, A, Savino, M 2007 A global survey of university ranking and league tables. Higher Education in Europe 32 1 515 .

Manuscript submission: http://www.editorialmanager.com/scim/

  • Impact Factor (2019): 2.867
  • Scimago Journal Rank (2019): 1.210
  • SJR Hirsch-Index (2019): 106
  • SJR Quartile Score (2019): Q1 Computer Science Apllications
  • SJR Quartile Score (2019): Q1 Library and Information Sciences
  • SJR Quartile Score (2019): Q1 Social Sciences (miscellaneous)
  • Impact Factor (2018): 2.770
  • Scimago Journal Rank (2018): 1.113
  • SJR Hirsch-Index (2018): 95
  • SJR Quartile Score (2018): Q1 Library and Information Sciences
  • SJR Quartile Score (2018): Q1 Social Sciences (miscellaneous)

For subscription options, please visit the website of Springer

Scientometrics
Language English
Size B5
Year of
Foundation
1978
Volumes
per Year
4
Issues
per Year
12
Founder Akadémiai Kiadó
Founder's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Publisher Akadémiai Kiadó
Springer Nature Switzerland AG
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
CH-6330 Cham, Switzerland Gewerbestrasse 11.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 0138-9130 (Print)
ISSN 1588-2861 (Online)