View More View Less
  • 1 INRA Lereco (SAE2) U1134, Rue de la Géraudière, BP 71627, 44316 Nantes Cedex 03, France
  • | 2 Observatoire des Sciences et des Techniques (OST), 75015 Paris, France
Restricted access

Abstract

J.K. Vanclay's article is a bold attempt to review recent works on the journal impact factor (JIF) and to call for alternative certifications of journals. The too broad scope did not allow the author to fulfill all his purposes. Attempting after many others to organize the various forms of criticism, with targets often broader than the JIF, we shall try to comment on a few points. This will hopefully enable us to infer in which cases the JIF is an angel, a devil, or a scapegoat. We shall also expand on a crucial question that Vanclay could not really develop in the reduced article format: the field-normalization. After a short recall on classical cited-side or ex post normalization and of the powerful influence measures, we will devote some attention to the novel way of citing-side or ex ante normalization, not only for its own interest, but because it directly proceeds from the disassembling of the JIF clockwork.

  • Adams, J, Gurney, K, Marshall, S 2007 Profiling citation impact: a new methodology. Scientometrics 72 2 325344 .

  • Adler, R, Ewing, J, Taylor, P 2008 Citation statistics. A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the International Council of Institute of Mathematical Statistics (IMS). Summarized in (2009). Statistical Science 24 1 114 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bar-Ilan, J 2008 Which h-index? – A comparison of WoS, Scopus and Google Scholar. Scientometrics 74 2 257271 .

  • Bergstrom, C. (2007). Eigenfactor: measuring the value and prestige of scholarly journals. College & Research Libraries News, 68, 5, www.ala.org/ala/acrl/acrlpubs/crlnews/backissues2007/may2007/eigenfactor.cfm.

    • Search Google Scholar
    • Export Citation
  • Bollen, J H Van de Sompel Hagberg, A, Chute, R 2009 A principal component analysis of 39 scientific impact measures. PLoS ONE 4 6 e6022 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bourdieu, P 1975 The specificity of the scientific field and the social conditions of the progress of reason. Social Science Information 14 6 1947 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bouyssou, D, Marchant, T 2011 Bibliometric rankings of journals based on impact factors: an axiomatic approach. Journal of Informetrics 5 1 7586 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Braun, T, Glänzel, W, Schubert, A 2006 A Hirsch-type index for journals. Scientometrics 69:169173 .

  • Callon, M, Latour, B 1981 Unscrewing the big leviathan: how actors macrostructure reality and how sociologists help them to do so Karin D Knorr Cetina Aaraon V Cicourel eds. Advances in social theory and methodology: toward an integration of micro- and macro-sociologies Routledge and Kegan Paul Boston 277303.

    • Search Google Scholar
    • Export Citation
  • Cronin, B 1984 The citation process; the role and significance of citations in scientific communication Taylor Graham London.

  • Czapski, G 1997 The use of deciles of the citation impact to evaluate different fields of research in Israel. Scientometrics 40 3 437443 .

  • de Moya-Anegon, F. (2007). SCImago. SJR — SCImago Journal & Country Rank.

  • DJ de Solla Price 1963 Little science, big science Columbia University Press New York.

  • Garfield, E 1955 Citation Indexes for Science. A new dimension in documentation through association of ideas. Science 122:108111 .

  • Garfield, E 1972 Citation analysis as a tool in journal evaluation. Science 178 4060 471479 .

  • Garfield, E 2006 The history and meaning of the journal impact factor. Journal of the American Medical Association 295:9093 .

  • Garfield, E, Sher, IH 1963 New factors in the evaluation of scientific literature through citation indexing. American Documentation 14 3 195201 .

  • Geller, NL 1978 Citation influence methodology of Pinski and Narin. Information Processing and Management 14:9395 .

  • Glänzel, W 2008 On some new bibliometric applications of statistics related to the h-index. Scientometrics 77 1 187196 .

  • Glanzel, W, Moed, HF 2002 Journal impact measures in bibliometric research. Scientometrics 53 2 171193 .

  • Glänzel, W, Schubert, A, Thijs, B, Debackere, K 2011 A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking. Scientometrics 87 2 415424 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hagström, WO 1965 The scientific community Basic Books New York.

  • Hicks, D 2004 The four literatures of social science H Moed W Glanzel U Schmoch eds. Handbook of quantitative science and technology research Kluwer Academic New York.

    • Search Google Scholar
    • Export Citation
  • Hirsch, JE 2007 Does the h index have predictive power?. Proceedings of the National Academy of Sciences 104 49 1919319198 .

  • Hoeffel, C 1998 Journal impact factors. Allergy 53:1225 .

  • Hönekopp, J, Khan, J 2011 Future publication success in science is better predicted by traditional measures than by the h index. Scientometrics 90 3 843853 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ingwersen, P, Larsen, B, Rousseau, R, Russell, J 2001 The publication-citation matrix and its derived quantities. Chinese Science Bulletin 46 6 524528 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Katz, SJ 1999 The self-similar science system. Research policy 28 5 501517 .

  • Leydesdorff, L, Opthof, T 2010 Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology 61 11 23652396 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lundberg, J 2007 Lifting the crown: citation z-score. Journal of Informetrics 1 2 145154 .

  • Luukkonen, T 1997 Why has Latour's theory of citations been ignored by the bibliometric community? Discussion of sociological interpretations of citation analysis. Scientometrics 38 1 2737 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marchant, T 2009 An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors. Scientometrics 80 2 325342 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marshakova-Shaikevich, I 1996 The standard impact factor as an evaluation tool of science fields and scientific journals. Scientometrics 35 2 283290 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Merton, R.K. (1942). Science and technology in a democratic order. Journal of legal and political sociology, 1, 115126 (reprint: The normative structure of science (1973). In Storer N.W. (ed.), The sociology of science: theoretical and empirical investigations (pp. 12671278). Chicago: University of Chicago Press).

    • Search Google Scholar
    • Export Citation
  • Moed, HF 2010 Measuring contextual citation impact of scientific journals. Journal of Informetrics 4 3 265277 .

  • Moed, HF TN van Leeuwen 1995 Improving the accuracy of institute for scientific information's journal impact factors. Journal of the American Society for Information Science 46 6 461467 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moed, HF, Vriens, M 1989 Possible inaccuracies occurring in citation analysis. Journal of Information Science 15:95107 .

  • Murugesan, P, Moravcsik, MJ 1978 Variation of the nature of citation measures with journal and scientific specialties. Journal of the American Society for Information Science 29 3 141155 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Narin, F 1976 Evaluative bibliometrics : the use of publication and citation analysis in the evaluation of scientific activity (Report prepared for the National Science Foundation, Contract NSF C-627) Computer Horizons Cherry Hill.

    • Search Google Scholar
    • Export Citation
  • Nicolaisen, J, Frandsen, TF 2008 The reference return ratio. Journal of Informetrics 2 2 128135 .

  • Palacios Huerta, I, Volij, O 2004 The Measurement of intellectual influence. Econometrica 72 3 963977 .

  • Pinski, G, Narin, F 1976 Citation influence for journal aggregates of scientific publications: theory, with application to the literature of physics. Information Processing and Management 12:297312 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Raddichi, F, Fortunato, S, Castellano, C 2008 Universality of citation distributions: towards an objective measure of citation impact. Proceedings of the National Academy of Sciences 105 45 1726817272 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ramanana-Rahary, S, Zitt, M, Rousseau, R 2009 Aggregation properties of relative impact and other classical indicators: convexity issues and the Yule-Simpson paradox. Scientometrics 79 1–2 311327 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rousseau, R 2008 Woeginger's axiomatisation of the h-index and its relation to the g-index, the h(2)-index and the r2-index. Journal of Informetrics 2 4 335340 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rousseau, R, Egghe, L 2003 A general framework for relative impact indicators. Canadian Journal of Information and Library Science 27 1 2948.

    • Search Google Scholar
    • Export Citation
  • Schubert, A, Braun, T 1986 Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics 9 5–6 281291 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Seglen, PO 1992 The skewness of science. Journal of the American Society for Information Science 43:628638 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sen, BK 1992 Documentation Note Normalized Impact Factor. Journal of Documentation 48 3 318325 .

  • Small, H, Sweeney, E 1985 Clustering the science citation index using co-citations: 1. A comparison of methods. Scientometrics 7 3–6 391409 .

  • AFJ Van Raan 2000 On growth, ageing, and fractal differentiation of science. Scientometrics 47 2 347362 .

  • AFJ van Raan 2001 Competition amongst scientists for publication status: toward a model of scientific publication and citation distributions. Scientometrics 51 1 347357 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vanclay, JK 2009 Bias in the journal impact factor. Scientometrics 78 1 312 .

  • Vanclay, JK 2011 Impact Factor: outdated artefact or stepping-stone to journal certification?. Scientometrics .

  • Vieira, ES, Gomez, JANF 2011 The journal relative impact: an indicator for journal assessment. Scientometrics 89 2 631651 .

  • Vinkler, P 2002 Subfield problems in applying the Garfield (impact) factors in practice. Scientometrics 53 2 267279 .

  • Waltman, L NJ van Eck TN van Leeuwen Visser, MS AFJ van Raan 2011 Towards a new crown indicator: some theoretical considerations. Journal of Informetrics 5:3747 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Waltman, L., & van Eck, N. J. (2009). A taxonomy of bibliometric performance indicators based on the property of consistency. Proceedings of the 12th International Conference on Scientometrics and Informetrics, 1002-1003.

    • Search Google Scholar
    • Export Citation
  • Wouters, P 1997 Citation cycles and peer review cycles. Scientometrics 38 1 3955 .

  • Zitt, M 2010 Citing-side normalization of journal impact: a robust variant of the audience factor. Journal of Informetrics 4 3 392406 .

  • Zitt, M 2011 Behind citing-side normalization of citations: some properties of the journal impact factor. Scientometrics 89 1 329344 .

  • Zitt, M, Ramanana-Rahary, S, Bassecoulard, E 2003 Correcting glasses help fair comparisons in international science landscape: country indicators as a function of ISI database delineation. Scientometrics 56 2 259282 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zitt, M, Ramanana-Rahary, S, Bassecoulard, E 2005 Relativity of citation performance and excellence measures: from cross-field to cross-scale effects of field-normalisation. Scientometrics 63 2 373401 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zitt, M, Small, H 2008 Modifying the journal impact factor by fractional citation weighting: the audience factor. Journal of the American Society for Information Science and Technology 59 11 18561860 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zucker, LG, Darby, MR 1996 Star scientists and institutional transformation: patterns of invention and innovation in the formation of the biotechnology industry. Proceedings of the National Academy of Sciences 93 23 1270912716 .

    • Crossref
    • Search Google Scholar
    • Export Citation