View More View Less
  • 1 Research Center for Bibliometrics and Science Evaluation, Chinese Academy of Social Sciences, Beijing 100732, China
  • 2 College of Information Science and Technology, Drexel University, Philadelphia, PA 19104, USA; xuemei.gong@drexel.edu
Restricted access

Abstract

An index system for evaluating academic papers is constructed and verified based on the empirical analysis of papers that has gained the 6th Chinese Academy of Social Sciences Award for Outstanding Achievements. Some new index, such as paper discipline impact factor, discipline average cited rate per paper and discipline average downloaded rate per paper have been put forward in this paper. The empirical research results show that the ranking of papers calculated by this evaluation index system is in conformity with the awards determined by peer review in general, but still needs to be verified and improved in practice.

  • Dalkey, N, Helmer, O. An experimental application of the Delphi method to the use of experts. Management Science 1963 9 3 458467 .

  • Finkenstaedt, T. Measuring research performance in the humanities. Scientometrics 1990 19 5–6 409417 .

  • Forman, EH, Gass, SI. The analytical hierarchy process—an exposition. Operations Research 2001 49 4 469486 .

  • Glänzel, W, Thijs, B, Schubert, A, Debackere, K. Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance. Scientometrics 2009 78 1 165188 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hanney, S, Frame, I, Grant, J, Buxton, M, Young, T, Lewison, G. Using categorizations of citations when assessing the outcomes from health research. Scientometrics 2005 65 3 357379 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jiang, X, Yin, G, Mo, Z 2009 A Guide to the Humanities & Social Science Core Journals in China (2008 Edition) Social Sciences Academic Press Beijing.

    • Search Google Scholar
    • Export Citation
  • Landeta, J. Current validity of the Delphi method in social sciences. Technological Forecasting and Social Change 2006 73 5 467482 .

  • Moed, HF, Luwel, M, Nederhof, AJ. Towards research performance in the humanities. Library Trends 2002 50 3 498520.

  • Norris, M, Oppenheim, C. Citation counts and the research assessment exercise V. Archaeology and the 2001 RAE. Journal of Documentation 2003 59 6 709730 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Saaty, TL. A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 1977 15 3 234281 .

  • Skulmoski, GJ, Hartman, FT, Krahn, J. The Delphi method for graduate research. The Journal of Information Technology Education 2007 6:121.

    • Search Google Scholar
    • Export Citation
  • Vidal, L. A., Marle, F., & Bocquet, J. C. (2010). Using a Delphi process and the analytic hierarchy process (AHP) to evaluate the complexity of projects. Expert Systems with Applications. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhang, L, Zhao, H, Li, Q, Wang, J, Tan, X. Establishment of paper assessment system based on academic disciplinary benchmarks. Scientomerics 2010 84:421429 .

    • Crossref
    • Search Google Scholar
    • Export Citation