View More View Less
  • 1 Laboratory for Studies of Research and Technology Transfer, School of Engineering, Dipartimento di Ingegneria dell’Impresa, Università degli Studi di Roma “Tor Vergata”, Via del Politecnico 1, 00133, Rome, Italy dangelo@disp.uniroma2.it
  • 2 National Research Council of Italy, P.le Aldo Moro 7, 00185, Rome, Italy
Restricted access

Abstract

There is an evident and rapid trend towards the adoption of evaluation exercises for national research systems for purposes, among others, of improving allocative efficiency in public funding of individual institutions. However the desired macroeconomic aims could be compromised if internal redistribution of government resources within each research institution does not follow a consistent logic: the intended effects of national evaluation systems can result only if a “funds for quality” rule is followed at all levels of decision-making. The objective of this study is to propose a bibliometric methodology for: (i) large-scale comparative evaluation of research performance by individual scientists, research groups and departments within research institution, to inform selective funding allocations; and (ii) assessment of strengths and weaknesses by field of research, to inform strategic planning and control. The proposed methodology has been applied to the hard science disciplines of the Italian university research system for the period 2004–2006.

  • Abramo, G, D’Angelo, CA, Di Costa, F 2008 Assessment of sectoral aggregation distortion in research productivity measurements. Research Evaluation 17 2 111121 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Abramo, G, D’Angelo, CA, Caprasecca, A 2008 Gender differences in research productivity: A bibliometric analysis of the Italian academic system. Scientometrics 79 3 517539 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bhattacharya, A., & Newhouse, H. (2008). Allocative efficiency and an incentive scheme for research. University of California-San Diego Working Paper. Retrieved June 18, 2010 from http://econ.ucsd.edu/∼hnewhous/research/Bhattacharya-Newhouse-RAE.pdf.

    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Mutz, R, Neuhaus, C, Daniel, H-D 2008 Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics 8:93102 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Butler, L 2003 Explaining Australia's increased share of ISI publications. The effects of a funding formula based on publication counts. Research Policy 32 1 143155 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Costas, R T van Leeuwen Bordons, M 2010 A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact. Journal of the American Society for Information Science and Technology 61 8 15641581.

    • Search Google Scholar
    • Export Citation
  • ERA (2010). The Excellence in Research for Australia (ERA) Initiative. Retrieved June 18, 2010 from http://www.arc.gov.au/era/default.htm.

    • Search Google Scholar
    • Export Citation
  • Franceschet, M 2009 A cluster analysis of scholar and journal bibliometric indicators. Journal of the American Society for Information Science and Technology 60 10 19501964 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Geuna, A, Martin, BR 2003 University research evaluation and funding: an international comparison. Minerva 41:277304 .

  • Hicks, D 2009 Evolving regimes of multi-university research evaluation. Higher Education 57 4 393404 .

  • Hirsch, JE 2005 An index to quantify an individual's scientific research output. Proceedings of the National Academy of Science of the USA 102 46 1656916572 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Leydesdorff, L 2008 Caveats for the use of citation indicators in research and journal evaluations. Journal of the American Society for Information Science and Technology 59 2 278287 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • ORP (2009). Observatory on Public Research in Italy. Retrieved June 18, 2010 from www.orp.researchvalue.it.

  • Orr, D, Jaeger, M, Schwarzenberger, A 2007 Performance-based funding as an instrument of competition in German higher education. Journal of Higher Education Policy and Management 29 1 323 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • PBRF (2008). Performance-Based Research Fund in New Zealand. Retrieved June 18, 2010 from http://www.tec.govt.nz/templates/standard.aspx?id=588.

    • Search Google Scholar
    • Export Citation
  • RAE (2008). Research Assessment Exercise. Retrieved June 18, 2010 from www.rae.ac.uk.

  • REF (2010). Research Excellence Framework. Retrieved June 18, 2010 from http://www.hefce.ac.uk/Research/ref/.

  • Rousseau, R, Smeyers, M 2000 Output-financing at LUC. Scientometrics 47 2 379387 .

  • Sandström, U., & Sandström, E., (2009). Meeting the micro-level challenges: Bibliometrics at the individual level. 12th International Conference on Scientometrics and Informetrics, Rio de Janeiro, Brazil, July 14-17.

    • Search Google Scholar
    • Export Citation
  • Shattock, M 2004 Managing successful universities. Perspectives: Policy and Practice in Higher Education 8 4 119120 .

  • Strehl, F., Reisinger, S., & Kalatschan, M. (2007). Funding systems and their effects on higher education systems. OECD Education Working Papers, No. 6. OECD Publishing. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • H Van den Berghe Houben, JA RE de Bruin Moed, HF, Kint, A, Spruyt, EHJ, Luwel, M 1998 Bibliometric indicators of university research performance in Flanders. Journal of the American Society for Information Science 49 1 5967 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Th N Van Leeuwen Moed, HF 2002 Development and application of journal impact measures in the Dutch science system. Scientometrics 53:249266 .

  • VTR (2006). Italian Triennial Research Evaluation. VTR 2001-2003. Risultati delle valutazioni dei Panel di Area. Retrieved June 18, 2010 from http://vtr2006.cineca.it/.

    • Search Google Scholar
    • Export Citation

Manuscript submission: http://www.editorialmanager.com/scim/

  • Impact Factor (2019): 2.867
  • Scimago Journal Rank (2019): 1.210
  • SJR Hirsch-Index (2019): 106
  • SJR Quartile Score (2019): Q1 Computer Science Apllications
  • SJR Quartile Score (2019): Q1 Library and Information Sciences
  • SJR Quartile Score (2019): Q1 Social Sciences (miscellaneous)
  • Impact Factor (2018): 2.770
  • Scimago Journal Rank (2018): 1.113
  • SJR Hirsch-Index (2018): 95
  • SJR Quartile Score (2018): Q1 Library and Information Sciences
  • SJR Quartile Score (2018): Q1 Social Sciences (miscellaneous)

For subscription options, please visit the website of Springer