View More View Less
  • 1 Institute for Systems Analysis and Computer Science, National Research Council of Italy, Viale Manzoni, 30, 00185, Rome, Italy; dangelo@dii.uniroma2.it
  • 2 Laboratory for Studies of Research and Technology Transfer at University of Rome “Tor Vergata”, Dipartimento di Ingegneria dell'Impresa, Università degli Studi di Roma “Tor Vergata”, Via del Politecnico 1, 00133, Rome, Italy; tindaro.cicero@uniroma2.it
Restricted access

Abstract

National research assessment exercises are conducted in different nations over varying periods. The choice of the publication period to be observed has to address often contrasting needs: it has to ensure the reliability of the results issuing from the evaluation, but also reach the achievement of frequent assessments. In this work we attempt to identify which is the most appropriate or optimal publication period to be observed. For this, we analyze the variation of individual researchers’ productivity rankings with the length of the publication period within the period 2003–2008, by the over 30,000 Italian university scientists in the hard sciences. First we analyze the variation in rankings referring to pairs of contiguous and overlapping publication periods, and show that the variations reduce markedly with periods above 3 years. Then we will show the strong randomness of performance rankings over publication periods under 3 years. We conclude that the choice of a 3 year publication period would seem reliable, particularly for physics, chemistry, biology and medicine.

  • Abramo, G, Cicero, T, D'Angelo, CA. Assessing the varying level of impact measurement accuracy as a function of the citation window length. Journal of Informetrics 2011 5 4 659667 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Abramo, G, Cicero, T, D'Angelo, CA. A sensitivity analysis of researchers’ productivity rankings to citation window length. Journal of Informetrics 2012 6 2 192201 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Abramo, G, D'Angelo, CA. Measuring science: irresistible temptations, easy shortcuts and dangerous consequences. Current Science 2007 93 6 762766.

    • Search Google Scholar
    • Export Citation
  • Abramo, G, D'Angelo, CA. National-scale research performance assessment at the individual level. Scientometrics 2011 86 2 347364 .

  • Amat, CB. Editorial and publication delay of papers submitted to 14 selected food research journals. Influence of Online Posting. Scientometrics 2008 74 3 379389.

    • Search Google Scholar
    • Export Citation
  • Bhattacharya, A., Newhouse, H. (2010). Allocative efficiency and an incentive scheme for research. Discussion Papers 10/02, Department of Economics, University of York.

    • Search Google Scholar
    • Export Citation
  • Burrell, QL. Modeling citation age data: simple graphical methods from reliability theory. Scientometrics 2002 55:273285 .

  • Butler, L. Modifying publication practices in response to funding formulas. Research Evaluation 2003 17 1 3946 .

  • D'Angelo, CA, Giuffrida, C, Abramo, G. A heuristic approach to author name disambiguation in large-scale bibliometric databases. Journal of the American Society for Information Science and Technology 2011 62 2 257269 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • ERA (2010). The excellence in research for Australia (ERA) initiative. http://www.arc.gov.au/era/. Accessed 16 December 2011.

  • Geuna, A, Martin, BR. University research evaluation and funding: an international comparison. Minerva 2003 41 4 277304 .

  • Glänzel, W. Towards a model for diachronous and synchronous citation analyses. Scientometrics 2004 60 3 511522 .

  • Lundberg, J. Lifting the crown—citation z-score. Journal of Informetrics 2007 1 2 145154 .

  • Luwel, M, Moed, HF. Publication delays in the science field and their relationship to the ageing of scientific literature. Scientometrics 1998 41 1–2 2940 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moed, H. F. (2005). Citation analysis in research evaluation. Springer, ISBN:978-1-4020-3713-9.

  • OECD (2010). Performance-based funding for public research in tertiary education institutions. Workshop proceedings OECD 2010, 187, ISBN:978-92-64-09461-1.

    • Search Google Scholar
    • Export Citation
  • Rehn, C., Kronman, U., Wadsko, D. (2007). Bibliometric indicators definitions and usage at Karolinska Institutet. Karolinska Institutet University Library. http://kib.ki.se/sites/kib.ki.se/files/Bibliometric_indicators_definitions_1.0.pdf. Accessed 8 March 2012.

    • Search Google Scholar
    • Export Citation
  • Trivedi, PK. An analysis of publication lags in econometrics. Journal of Applied Econometrics 1993 8 1 93100 .