View More View Less
  • 1 Norwegian Institute for Studies in Research and Higher Education (NIFU), Hegdehaugsvn. 31, NO-0352 Oslo, Norway
Restricted access

Citation distributions are extremely skewed. This paper addresses the following question: To what extent are national citation indicators influenced by a small minority of highly cited articles? This question has not been studied before at the level of national indicators. Using the scientific production of Norway as a case, we find that the average citation rates in major subfields are highly determined by one or only a few highly cited papers. Furthermore, there are large annual variations in the influence of highly cited papers on the average citation rate of the subfields. We conclude that an analysis of the underlying data for national indicators may be useful in creating awareness towards the occurrence of particular articles with great influence on what is normally considered an indicator of “national performance”, and that the common interpretation of the indicator on research policy level needs to be informed by this fact.

  • 1. De Solla Price, D. J. 1965 Networks of scientific papers Science 149 510515.

  • 2. Seglen, P. O. 1992 The skewness of science Journal of the American Society for Information Science 43 628638.

  • 3. GlÄnzel, W. 2000 Science in Scandinavia: A bibliometric approach Scientometrics 48 121150.

  • 4. Moed, H. F. Van Leeuwen, T. N. Reedijk, J. 1999 Towards appropriate indicators of journal impact Scientometrics 46 575589.

  • 5. Garfield, E., The use of journal impact factors and citation analysis for evaluation of science, Paper presented at the 41st Annual Meeting of the Council of Biology Editors, Salt Lake City (1998), available at: http://www.garfield.library.upenn.edu/papers/eval_of_science_CBE(Utah).html.

    • Search Google Scholar
    • Export Citation
  • 6. National Science Board 2002 Science and Engineering Indicators — 2002 National Science Foundation Arlington.

  • 7. Aksnes, D. W., Characteristics of highly cited papers, Research Evaluation, 12(3) (2003).

  • 8. European Commission, Third European Report on Science & Technology Indicators 2003, Brussels 2003.

  • 9. Seglen, P. O. 1994 Causal relationship between article citedness and journal impact Journal of the American Society for Information Science 45 111.

    • Search Google Scholar
    • Export Citation
  • 10. Martin, B. R. Irvine, J. 1983 Assessing basic research: Some partial indicators of scientific progress in radio astronomy Research Policy 12 6190.

    • Search Google Scholar
    • Export Citation
  • 11. Plomp, R. 1994 The highly cited papers of professors as an indicator of a research group's scientific performance Scientometrics 29 377393.

    • Search Google Scholar
    • Export Citation
  • 12. Martin, B. R. 1996 The use of multiple indicators in the assessment of basic research Scientometrics 36 343362.

  • 13. Tijssen, R. J. W. Visser, M. S. Van Leeuwen, T. N. 2002 Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference? Scientometrics 54 381397.

    • Search Google Scholar
    • Export Citation
  • 14. European Commission, Key Figures 2001. Special Edition. Indicators for Benchmarking of National Research Policies. Brussels 2001.