View More View Less
  • 1 Research Center for Chinese Science Evaluation, Wuhan University, Wuhan 430072, China
Restricted access

Abstract

As indicator weights obtaining is often difficult in all types of evaluation, this paper describes an approach to improve the indicator weights of scientific and technological competitiveness evaluation of Chinese universities. As a public institution funded by Chinese government, the research center for Chinese science evaluation of Wuhan University has completed five annual evaluations for the scientific and technological competitiveness of Chinese universities since 2005, whose abundant and reliable data motivated us to try to improve the weights obtained by the AHP (analytical hierarchy process). Based on these data, we calculated the objective weights of the indicator using the representative mathematical methods of the least square and the variation coefficient. As the weights of AHP can be influenced by the knowledge, experience and preference of experts and the calculated objective weights neglect the subjective judgement information, we integrated the subjective and objective weights by respectively using the additive and multiplicative model to reflect both the subjective considerations of experts and the objective information, and obtained three kinds of integrative weights. Finally, we selected the integrative weights of multiplicative model as the best weights by comparing and analyzing the evaluation results in 2005 and 2009 of each kind of weights. The results show that the evaluation effect of the weights of multiplicative model is indeed the best for all types of Chinese universities among these kinds of weights, and the experts and university principals enquired also basically reached a consensus on the university rankings of the integrative weights of multiplicative model.

  • Bonnevie-Nebelong, E 2006 Methods for journal evaluation: Journal citation identity, journal citation image and internationalization. Scientometrics 66 2 411424 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Butler, L 2002 Identifying ‘highly-rated’ journals—an Australian case study. Scientometrics 53 2 207227 .

  • Diakoulaki, D, Mavrotas, G, Papayannakis, L 1992 A multicriteria approach for evaluating the performance of industrial firms. Omega 20:467474 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Garfield, E 1972 Citation analysis as a tool in journal evaluation. Science 178:471479 .

  • Guo, YJ 2007 The theory, method and application of comprehensive evaluation Science Press Beijing.

  • Higgins, JC 1989 Performance measurement in universities. European Journal of Operational Research 38:358368 .

  • Huang, MH, Chang, HW, Chen, DZ 2006 Research evaluation of research-oriented universities in Taiwan from 1993 to 2003. Scientometrics 67 3 419435.

    • Search Google Scholar
    • Export Citation
  • Hwang, CL, Lin, MJ 1987 Group decision making under multiple criteria: Methods and applications Springer Berlin.

  • Hwang, CL, Yoon, K 1981 Multiple attribute decision making: Methods and applications Springer Berlin.

  • Kao, C, Pao, HL 2009 An evaluation of research performance in management of 168 Taiwan universities. Scientometrics 78 2 261277 .

  • Ma, J, Fan, ZP, Huang, LH 1999 A subjective and objective integrated approach to determine attribute weights. European Journal of Operational Research 112:397404 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Madu, CN 1994 A quality confidence procedure for GDSS application in multicriteria decision making. IIE Transactions 26 3 3139 .

  • Mao, DX 2002 A combinational evaluation method resulting in consistency between subjective and objective evaluation in the least squares sense. Journal of Chinese Management Science 10 5 9597.

    • Search Google Scholar
    • Export Citation
  • Mcgrath, WE 1987 Ratings and rankings: Multiple comparisons of mean ratings. College and Research Libraries 48:169172.

  • Morrissey, AJ, Browne, J 2004 Waste management models and their application to sustainable waste management. Waste Management 24:297308 .

  • Qiu, JP 2005 The evaluation report of Chinese universities and specialties of 2005–2006 Science Press Beijing.

  • Qiu, JP 2006 The evaluation report of Chinese universities and specialties of 2006–2007 Science Press Beijing.

  • Qiu, JP 2007 The evaluation report of Chinese universities and specialties of 2007–2008 Science Press Beijing.

  • Qiu, JP 2008 The evaluation report of Chinese universities and specialties of 2008–2009 Science Press Beijing.

  • Qiu, JP 2009 The evaluation report of Chinese universities and specialties of 2009–2010 Science Press Beijing.

  • Raan, A 1999 Advanced bibliometric methods for the evaluation of universities. Scientometrics 45 3 418420 .

  • Saaty, TL 1977 A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 15 3 234281 .

  • Saaty, TL 1980 The analytic hierarchy process McGraw-Hill New York.

  • Saaty, TL 1990 Multicriteria decision making: The analytic hierarchy process RWS Publications Pittsburgh.

  • Thomson Scientific (2010). ESI v2.0 Reference Card. Retrieved January 24, 2010 from http://scientific.thomson.com/media/scpdf/esi-0805-q.pdf.

    • Search Google Scholar
    • Export Citation
  • Zeleny, M 1982 Multiple criteria decision making McGraw-Hill NY.

Manuscript submission: http://www.editorialmanager.com/scim/

  • Impact Factor (2019): 2.867
  • Scimago Journal Rank (2019): 1.210
  • SJR Hirsch-Index (2019): 106
  • SJR Quartile Score (2019): Q1 Computer Science Apllications
  • SJR Quartile Score (2019): Q1 Library and Information Sciences
  • SJR Quartile Score (2019): Q1 Social Sciences (miscellaneous)
  • Impact Factor (2018): 2.770
  • Scimago Journal Rank (2018): 1.113
  • SJR Hirsch-Index (2018): 95
  • SJR Quartile Score (2018): Q1 Library and Information Sciences
  • SJR Quartile Score (2018): Q1 Social Sciences (miscellaneous)

For subscription options, please visit the website of Springer

Scientometrics
Language English
Size B5
Year of
Foundation
1978
Volumes
per Year
4
Issues
per Year
12
Founder Akadémiai Kiadó
Founder's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Publisher Akadémiai Kiadó
Springer Nature Switzerland AG
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
CH-6330 Cham, Switzerland Gewerbestrasse 11.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 0138-9130 (Print)
ISSN 1588-2861 (Online)