View More View Less
  • 1 Department of Information and Computer Sciences, University of Hawaii, Honolulu, HI 96822, USA
Restricted access

Abstract

This paper reflects on the most current and some of the recent contributions of JK Vanclay, focusing on his methods, findings, and criticism about the journal citations reports and the web of science databases, the journal impact factor and the h-index. It is argued and demonstrated that some of the recent papers of the author about scientometric issues, measures and sources show so much demagoguery, ignorance and arrogance, have so much prejudice and bias, so profound errors in using the databases, calculating metrics, and interpreting search results that the papers are very unlikely to be meant as a genuine contribution from an academic who is a graduate of—among others—Oxford University, professor and dean in a respected university, a well-published and well-cited author and a recipient of the Queen's Award (all the above in forest science). The papers are much more likely to serve as props for a staged, mock-up scenario based on slipshod research in an experiment, to illustrate the deficiencies in the processes and in the assessment of scholarly publishing productivity and impact in order to present the idealized solution of Vanclay: using the h-index, portrayed as the Prince, mounted on the shoulder of the White Horse, Google Scholar.

  • Abraham, P 2000 Duplicate and salami publications. Journal of Postgraduate Medicine 46 2 6769.

  • Alonso, S, Cabrerizo, F, Herrera-Viedma, E, Herrera, F 2009 H-index: a review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics 3 4 273289 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Balaban, A. T. (2012). Positive and negative aspects of citation indices and journal impact factors. Scientometrics . doi: .

  • Bar-Ilan, J. (2008). Which h-index?—A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2), 257-271. Accessed April 3, 2012, from http://150.214.190.154/hindex/pdf/Bar-Ilan2008.pdf.

    • Search Google Scholar
    • Export Citation
  • Bar-Ilan, J 2010 Ranking of information and library science journals by JIF and by h-type indices. Journal of Informetrics 4 2 141147 .

  • Bensman, SJ 1998 Scientific and technical serials holdings optimization in an inefficient market: A LSU serials redesign project exercise. Library Resources & Technical Services 42 3 147242.

    • Search Google Scholar
    • Export Citation
  • Bensman, SJ 2007 Garfield and the impact factor: The creation, utilization, and validation of a citation measure. Annual Review of Information Science and Technology 41 1 93155 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bensman, S. J. (2011). The impact factor: its place in Garfield's thought in science evaluation and in library collection. Scientometrics. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bollen, J, Rodriquez, MA H Van De Sompel 2006 Journal status. Scientometrics 69 3 669687 .

  • Bornmann, L, Mutz, R, Neuhaus, C, Daniel, HD 2008 Citation counts for research evaluation: Standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics 8 1 93102 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Rüdiger, M, Daniel, H-D 2009 Do we need the h index and its variants in addition to standard bibliometric measures?. Journal of the American Society for Information Science and Technology 60 6 12861289 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Braun, T. (2007). The impact factor of scientific and scholarly journals: Its use and misuse in research evaluation. Akadémiai Kiadó, Budapest. ISBN 963-05-8438-7.

    • Search Google Scholar
    • Export Citation
  • Braun, T, Glänzel, W 1995 On a source of errors in computing journal impact factors. Chemical Intelligencer 1:3132.

  • Braun, T, Glänzel, W, Schubert, A 2005 A Hirsch-type index for journals. The Scientist 19 11 8.

  • Brumback, RA 2009 Impact factor wars: Episode V.: The empire strikes back. Journal of Child Neurology 24 3 260262 .

  • Brumback, RA 2009 Impact factor: Let's be unreasonable. Epidemiology 20 6 932933 .

  • Buchanan, RA 2006 Accuracy of cited references: The role of citation databases. College and Research Libraries 67:292303.

  • Butler, L 2011 The devil is in the detail: Concerns about Vanclay's analysis of Australian journal rankings. Journal of Informetrics 5 4 693694 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Campanario, JM 2011 Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window. Scientometrics 87 1 189204 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dong, P, Loh, M, Mondry, A 2005 The “impact factor” revisited. Biomedical Digital Libraries 2 7 18.

  • Falagas, ME, Alexiou, VG 2008 The top-ten in journal impact factor manipulation. Archivum Immunologiae et Therapiae Experimentalis 56 4 223226 .

  • Garfield, E. (1994). The Thomson Reuters impact factor. Accessed April 3, 2012, from http://thomsonreuters.com/products_services/science/free/essays/impact_factor/.

    • Search Google Scholar
    • Export Citation
  • Garfield, E 1996 How can impact factors be improved?. British Medical Journal 313:411413 .

  • Garfield, E 1999 Journal impact factor: A brief review. Canadian Medical Association Journal 161 8 979980.

  • Garfield, E. (2005). The agony and the ecstasy—The history and meaning of the journal impact factor. Chicago: International Congress on Peer Review and Biomedical Publication.

    • Search Google Scholar
    • Export Citation
  • Garfield, E 2006 The history and meaning of the journal impact factor. Journal of the American Medical Association 295 1 9093 .

  • Glänzel, W 2009 The multi-dimensionality of journal impact. Scientometrics 78 2 355374 .

  • Gorman, GE 2008 They can't read, but they sure can count—Flawed rules of the journal rankings game. Online Information Review 32 6 705708 .

  • Harnad, S 2008 Validating research performance metrics against peer rankings. Ethics in Science and Environmental Politics 8 1 103107 .

  • Hirsch, JE 2005 An index to quantify an individual's scientific research output. Proceedings of the National Academies of Science 102 46 1656916572 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hirst, G 1978 Discipline impact factors: A method for determining core journal lists. Journal of the American Society for Information Science 29 4 171172 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jacso, P 2000 The number game. Online Information Review 24 2 180183.

  • Jacso, P 2001 A deficiency in the algorithm for calculating the impact factor of scholarly journals: The journal impact factor. Cortex 37 4 590594 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jacso, P 2004 WorldCat, Sociological Abstracts on CSA. e-Psyche 28 2 5458.

  • Jacso, P. (2005a). Google Scholar (Redux). Gale Reference Reviews (online), June 2005. Accessed March 31, 2012, from http://www.gale.cengage.com/reference/peter/index.htm.

    • Search Google Scholar
    • Export Citation
  • Jacso, P 2005 Google Scholar: The pros and the cons. Online Information Review 29 2 208214 .

  • Jacso, P 2006 Deflated, inflated and phantom citation counts. Online Information Review 30 3 297309 .

  • Jacso, P 2007 How big is a database versus how is a database big. Online Information Review 31 4 533536 .

  • Jacso, P 2007 Software issues related to cited references. Online Information Review 31 6 892905 .

  • Jacso, P 2007 The dimensions of cited reference enhanced database subsets. Online Information Review 31 5 695705 .

  • Jacso, P. (2007d). Trends in professional and academic online information services. (Keynote Address) at INFORUM 13th conference on professional information resources prague May 22-24, 2007. Accessed April 3, 2012, from http://www.inforum.cz/pdf/2007/jacso-peter.pdf.

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2008a). Google Scholar revisited. Online Information Review, 32 (1), 102114. Accessed March 31, 2012, from http://www.jacso.info/OIR/2008-01-jacso-Google-Scholar-revisited.doc (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Jacso, P 2008 Testing the calculation of a realistic h-index in Google Scholar, Scopus and Web of Science for F.W. Lancaster. Library Trends 56 4 784815 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2008c). The plausibility of computing the h-index of scholarly productivity and impact using reference-enhanced databases. Online Information Review, 32 (2), 266283. Accessed April 3, from, http://www.jacso.info/PDFs/jacso-plausibility-of-computing-the-h-index-jav.pdf.

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2008d). The pros and cons of computing the h-index using Google Scholar. Online Information Review, 32 (3), 437452. Accessed March 31, 2012, from http://www.jacso.info/PDFs/jacso-GS-for-h-index-unabridged.pdf (unabridged manuscript).

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2008e). The pros and cons of computing the h-index using Web of Science. Online Information Review, 32 (5), 673688. Accessed March 31, 2012, from http://www.jacso.info/OIR/2008-05-jacso-pros-and-cons-of-computing-the-h-index-using-Web-of-Science.doc (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2009a). Calculating the h-index and other bibliometric and scientometric indicators from Google Scholar with the Publish or Perish software. Online Information Review, 33 (6), 11891200. Accessed March 31, 2012, from http://www.jacso.info/OIR/2009-06-jacso-calculating-H-index-and-other-bibliometric-and-scientometric-indicators.doc (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2009b). Database source coverage: hypes, vital signs and reality checks. Online Information Review, 33 (5), 9971007. Accessed March 31, 2012, from http://www.jacso.info/OIR/2009-05-jacso-database-source-coverage-hypes-and-reality-checks.doc (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2009c). Errors of omission and their implication for computing scientometric measures in evaluating the publishing productivity and impact of countries. Online Information Review, 33 (2), 376385. Accessed March 31, 2012, from http://www.jacso.info/OIR/2009-02-jacso-errors-of-omission-and-their-implications-for-computing-scientometrics-measures.doc (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2009d). Five-year impact factor data in the Journal Citation Reports. Online Information Review, 33 (3), 603614. Accessed February 5, 2012, from http://www.jacso.info/OIR/2009-03-five-year-impact-factor-data-in-the-Journal-Citation-Reports.doc (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2009e). Google Scholar's ghost authors. Library Journal, 134 (18), 2627. Accessed April 3, 2012, from http://www.libraryjournal.com/article/CA6698580.html?q=jacso.

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2009f). The h-index for countries in Web of Science and Scopus. Online Information Review, 33 (4), 831837. Accessed March 31, 2012, from http://www.jacso.info/OIR/2009-04-jacso-h-index-for-countries-in-WoS-and-Scopus.doc (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2009g). Web of Science. Gale Reference Reviews (online), July 2009. Accessed March 31, 2012, from http://www.gale.cengage.com/reference/peter/200907/science.html.

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2010a). Eigenfactor and Article Influence Score in the Journal Citation Reports. Online Information Review, 34 (2), 339348. Accessed March 31, 2012, from http://www.jacso.info/jcr-eigenfactor/.

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2010b). Metadata mega mess in Google Scholar. Online Information Review, 34 (1), 175191. Accessed March 31, 2012, from http://www.jacso.info/OIR/2010-01-jacso-metadata-mega-mess-in-Google-Scholar.doc.

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2011). Google Scholar duped and deduped—the aura of ‘robometrics’. Online Information Review, 35 (1), 154160. Accessed March, 31, 2012 from http://www.jacso.info/OIR/2011-01-jacso-Google-Scholar-duped-deduped.doc (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2012a). Google Scholar author citation tracker: Is it too little, too late? Online Information Review, 36 (1), 126141. Accessed March 31, 2012, from http://www.jacso.info/OIR/2012-01-jacso-Google-Scholar-author-citation-tracker.doc (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Jacso, P. (2012b). Using Google Scholar for journal impact factors and the h-index in nationwide publishing assessments in academia—Siren songs and air-raid sirens. Online Information Review, 36 (3), (in press).

    • Search Google Scholar
    • Export Citation
  • Martin, BR 1996 The use of multiple indicators in the assessment of basic research. Scientometrics 36 3 343362 .

  • Mc Veigh, M, Mann, SJ 2009 The journal impact factor denominator: Defining citable (countable) items. Journal of the American Medical Association 302 10 11071109 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meho, LI, Yang, K 2007 Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology 58 13 21052125 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4 (3), 265277. Accessed April 3, 2012, from http://arxiv.org/ftp/arxiv/papers/0911/0911.2632.pdf (pre-print version).

    • Search Google Scholar
    • Export Citation
  • Moed, HF, Plume, A 2011 The multi-dimensional research assessment matrix. Research Trends 23:57.

  • Moed, HF TN Van Leeuwen 1995 Improving the accuracy of (the) Institute for Scientific Information's journal impact factors. Journal of the American Society for Information Science and Technology 46 6 461467 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moed, HF TN Van Leeuwen Reedijk, J 1996 A critical analysis of the journal impact factors of Angewandte Chemie and the Journal of the American Chemical Society: Inaccuracies in published impact factors based on overall citations only. Scientometrics 37 1 105116 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moed, HF TN Van Leeuwen Reedijk, J 1999 Towards appropriate indicators of journal impact. Scientometrics 46 3 575589 .

  • Nisonger, TE 1994 A methodological issue concerning the use of Social Sciences Citation Index Journal Citation Reports impact factor data for journal ranking. Library Collections, Acquisition and Technical Services 18 4 447458.

    • Search Google Scholar
    • Export Citation
  • Nisonger, TE 2004 The benefits and drawbacks of impact factor for journal collection management in libraries. Serials Librarian 47 1–2 5775 .

  • Nunberg, G. (2009). Google's book search: A disaster for scholars. The Chronicle of Higher Education, August 31. Accessed April 3, 2012, from http://chronicle.com/article/Googles-Book-Search-A/48245/.

    • Search Google Scholar
    • Export Citation
  • Oppenheim, C. (1996). Do citations count? Citation indexing and the Research Assessment Exercise (RAE). Serials: The Journal for the Serials Community, 9 (2), 155161.

    • Search Google Scholar
    • Export Citation
  • Opthof, T 1999 Submission acceptance rate rapid review system and impact factor. Cardiovascular Research 41 1 14 .

  • Pendlebury, DA 2009 The use and misuse of journal metrics and other citation indicators. Archivum Immunologiae Et Therapiae Experimentalis 57 1 111 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pendlebury, DA, Adams, J 2012 Comments on a critique of the Thomson Reuters journal impact factor. Scientometrics .

  • Pinski, G, Narin, F 1976 Citation influence for journal aggregates of scientific publications: Theory with application to the literature of physics. Information Processing and Management 12 5 297312 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pudovkin, AI, Garfield, E 2004 Rank-normalized impact factor: A way to compare journal performance across subject categories. Proceedings of the ASIST Annual Meeting 41:507515.

    • Search Google Scholar
    • Export Citation
  • Pudovkin, A. I., & Garfield, E. (2012). Rank normalization of impact factors will resolve Vanclay's dilemma with TRIF. Scientometrics. doi: .

  • Rousseau, R 2001 Journal evaluation: Technical and practical issues. Library Trends 50 3 418439.

  • Rousseau, R 2005 Median and percentile impact factors: A set of new indicators. Scientometrics 63 3 431441 .

  • Schubert, A, Braun, T 1996 Cross-field normalization of scientometric indicators. Scientometrics 36 3 311324 .

  • Schubert, A, Glänzel, W 1983 Statistical reliability of comparisons based on the citation impact of scientific publications. Scientometrics 5 1 5973 .

  • Seglen, PO 1997 Why the impact factor of journals should not be used for evaluating research. British Medical Journal 314:497502 .

  • Sombatsompop, N, Markpin, T, Premkamolnetr, N 2004 A modified method for calculating the impact factors of journals in ISI journal citation reports: Polymer science category in 1997–2001. Scientometrics 60 2 217235 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vanclay, J. K. (2008a). Gauging the impact of journals. Forest Ecology and Management, 256 (4), 507509. Accessed April 3, 2012, from http://epubs.scu.edu.au/cgi/viewcontent.cgi?article=1321&context=esm_pubs.

    • Search Google Scholar
    • Export Citation
  • Vanclay, J. K. (2008b). Ranking forestry journals using the h-index. Journal of Informetrics, 2 (4), 326334. Accessed March 31, 2012, from http://arxiv.org/ftp/arxiv/papers/0712/0712.1916.pdf.

    • Search Google Scholar
    • Export Citation
  • Vanclay, J. K. (2009). Bias in the journal impact factor. Scientometrics, 78 (1), 312. Accessed March 31, 2012, from http://arxiv.org/ftp/cs/papers/0612/0612091.pdf.

    • Search Google Scholar
    • Export Citation
  • Vanclay, J. K. (2011). An evaluation of the Australian Research Council's journal ranking. Journal of Informetrics, 5 (2), 265274. Accessed April 3, 2012, from http://arxiv.org/ftp/arxiv/papers/1009/1009.3359.pdf.

    • Search Google Scholar
    • Export Citation
  • Vanclay, J. K. (2012). Impact factor: Outdated artefact or stepping-stone to journal certification? Scientometrics, (online). Accessed March 31, 2012, from http://arxiv.org/ftp/arxiv/papers/1201/1201.3076.pdf.

    • Search Google Scholar
    • Export Citation
  • Williams, G. (2007). Should we ditch impact factors? British Medical Journal, 334 (7593), 568. Accessed April 3, 2012, from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1828313/.

    • Search Google Scholar
    • Export Citation
  • Wilson, CS 1999 Informetrics. Annual Review of Information Science and Technology 34:107247.

  • Yin, C-Y, Aris, MJ, Chen, X 2010 Combination of Eigenfactor™ and h-index to evaluate scientific journals. Scientometrics 84 3 639648 .

  • Yu, G, Wang, L 2007 The self-cited rate of scientific journals and the manipulation of their impact factors. Scientometrics 73 3 321330 .

  • Yu, G, Wang, X-H, Yu, D-R 2005 The influence of publication delays on impact factors. Scientometrics 64 2 235246 .

  • Impact Factor (2019): 2.867
  • Scimago Journal Rank (2019): 1.210
  • SJR Hirsch-Index (2019): 106
  • SJR Quartile Score (2019): Q1 Computer Science Apllications
  • SJR Quartile Score (2019): Q1 Library and Information Sciences
  • SJR Quartile Score (2019): Q1 Social Sciences (miscellaneous)
  • Impact Factor (2018): 2.770
  • Scimago Journal Rank (2018): 1.113
  • SJR Hirsch-Index (2018): 95
  • SJR Quartile Score (2018): Q1 Library and Information Sciences
  • SJR Quartile Score (2018): Q1 Social Sciences (miscellaneous)

Manuscript submission: http://www.editorialmanager.com/scim/