View More View Less
  • 1 School of Environmental Science and Management, Southern Cross University, P.O. Box 157, Lismore, NSW 2480, Australia
Restricted access

Abstract

A review of Garfield's journal impact factor and its specific implementation as the Thomson Reuters impact factor reveals several weaknesses in this commonly-used indicator of journal standing. Key limitations include the mismatch between citing and cited documents, the deceptive display of three decimals that belies the real precision, and the absence of confidence intervals. These are minor issues that are easily amended and should be corrected, but more substantive improvements are needed. There are indications that the scientific community seeks and needs better certification of journal procedures to improve the quality of published science. Comprehensive certification of editorial and review procedures could help ensure adequate procedures to detect duplicate and fraudulent submissions.

  • Abraham, P 2000 Duplicate and salami publications. Journal of Postgraduate Medicine 46:6769.

  • Abramo, G, D’Angelo, CA, Di Costa, F 2010 Citations versus journal impact factor as proxy of quality: Could the latter ever be preferable?. Scientometrics 84:821833.

    • Search Google Scholar
    • Export Citation
  • Adler, R, Ewing, J, Taylor, P 2008 Joint committee on quantitative assessment of research: Citation statistics. The Australian Mathematical Society Gazette 35 3 166188.

    • Search Google Scholar
    • Export Citation
  • Althouse, BM, West, JD, Bergstrom, CT, Bergstrom, T 2009 Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology 60:2734.

    • Search Google Scholar
    • Export Citation
  • Altmann, KG, Gorman, GE 1998 The usefulness of impact factors in serial selection: A rank and mean analysis using ecology journals. Library Acquisitions: Practice and Theory 22 2 147159.

    • Search Google Scholar
    • Export Citation
  • Andrade, A, González-Jonte, R, Campanario, JM 2009 Journals that increase their impact factor at least fourfold in a few years: The role of journal self-citations. Scientometrics 80:515528.

    • Search Google Scholar
    • Export Citation
  • Anonymous 2004 Complacency about misconduct. Nature 427:1.

  • Anonymous (2011). The Thomson Reuters journal selection process. Thomson Reuters, http://thomsonreuters.com/products_services/science/free/essays/journal_selection_process/. Accessed 2011 September 5.

    • Search Google Scholar
    • Export Citation
  • ARC (2010). The excellence in research for Australia (ERA) Initiative. Australian Researh Council, Canberra. http://www.arc.gov.au/era/default.htm. Accessed 2011 August 5.

    • Search Google Scholar
    • Export Citation
  • Archambault, E, Lariviere, V 2009 History of the journal impact factor: Contingencies and consequences. Scientometrics 79:635649.

  • Awrey, J, Inaba, K, Barmparas, G, Recinos, G, Teixeira, PGR, Chan, LS, Talving, P, Demetriades, D 2011 Reference accuracy in the general surgery literature. World Journal of Surgery 35:475479.

    • Search Google Scholar
    • Export Citation
  • Bain, CR, Myles, PS 2005 Relationship between journal impact factor and levels of evidence in anaesthesia. Anaesthesia and Intensive Care 33 5 567570.

    • Search Google Scholar
    • Export Citation
  • Barbour, V 2010 How ghost-writing threatens the credibility of medical knowledge and medical journals. Haematologica 95:12.

  • Bath, FJ, Owen, VE, Bath, PMW 1998 Quality of full and final publications reporting acute stroke trials: A systematic review. Stroke 29 10 22032210.

    • Search Google Scholar
    • Export Citation
  • Beal, J. (2011). Wiley-Blackwell announces continued growth in impact factor journals. Press release 11 July 2011, http://www.eurekalert.org/pub_releases/2011-07/w-wac071111.php. Accessed 2011 August 25.

    • Search Google Scholar
    • Export Citation
  • Bensman, SJ 2007 Garfield and the impact factor. Annual Review of Information Science and Technology 41:93155.

  • Bensman, SJ 2008 Distributional differences of the impact factor in the sciences versus the social sciences: An analysis of the probabilistic structure of the 2005 journal citation reports. Journal of the American Society for Information Science and Technology 59:13661382.

    • Search Google Scholar
    • Export Citation
  • Bensman, SJ, Smolinsky, LJ, Pudovkin, AI 2010 Mean citation rate per article in mathematics journals: Differences from the scientific model. Journal of the American Society for Information Science and Technology 61:14401463.

    • Search Google Scholar
    • Export Citation
  • Bernstam, EV, Herskovic, JR, Aphinyanaphongs, Y, Aliferis, CF, Sriram, MG, Hersh, WR 2006 Using citation data to improve retrieval from MEDLINE. Journal of the American Medical Informatics Association 13:96105.

    • Search Google Scholar
    • Export Citation
  • Berquist, TH 2008 Duplicate publishing or journal publication ethics 101. American Journal of Roentgenology 191:311312.

  • Bollen, J H Van de Sompel Hagberg, A, Chute, R 2009 A principal component analysis of 39 scientific impact measures. PLoS ONE 4 6 e6022.

  • Bonillo Perales, A 2002 Spanish pediatric publications in pubmed between 1996 and 2001 [Publicaciones pediátricas españolas en PubMed en los años 1996 y 2001]. Anales Espanoles de Pediatria 57 2 152156.

    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Nast, I, Daniel, H-D 2008 Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics 77:415432.

    • Search Google Scholar
    • Export Citation
  • Braun, T. (ed.) (2007). The Impact Factor of scientific and scholarly journals: Its use and misuse in research evaluation. Scientometrics Guidebooks Series (vol. 2, pp. 686). Budapest: Akademiai Kiado.

    • Search Google Scholar
    • Export Citation
  • Braun, T, Glanzel, W 1995 On a source of error in computing journal impact factors. Chemical Intelligencer 1:3132.

  • Brown, H 2007 How impact factors changed medical publishing: and science. BMJ 334:561564.

  • Brumback, RA 2009 Impact factor wars: Episode V: The empire strikes back. Journal of Child Neurology 24:260262.

  • Brumback, RA 2009 Impact factor: Let's be unreasonable. Epidemiology 20:932933.

  • Butakov, S, Scherbinin, V 2009 The toolbox for local and global plagiarism detection. Computers & Education 52:781788.

  • Callaham, M, McCulloch, C 2011 Longitudinal trends in the performance of scientific peer reviewers. Annals of Emergency Medicine 57:141148.

    • Search Google Scholar
    • Export Citation
  • Callaham, M, Wears, RL, Weber, E 2002 Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA 287 21 28472850.

    • Search Google Scholar
    • Export Citation
  • Calver, MC, Bradley, JS 2010 Patterns of citations of open access and non-open access conservation biology journal papers and book chapters. Conservation Biology 24 3 872880.

    • Search Google Scholar
    • Export Citation
  • Cameron, BD 2005 Trends in the usage of ISI bibliometric data: Uses, abuses, and implications. Portal 5:105125.

  • Campanario, JM 2011 Empirical study of journal impact factors obtained using the classical two-year citation window versus a 5-year citation window. Scientometrics 87:189204.

    • Search Google Scholar
    • Export Citation
  • Campanario, JM 2011 Large increases and decreases in journal impact factors in only 1 year: The effect of journal self-citations. Journal of the American Society for Information Science and Technology 62:230235.

    • Search Google Scholar
    • Export Citation
  • Campanario, JM, Carretero, J, Marangon, V, Molina, A, Ros, G 2011 Effect on the journal impact factor of the number and document type of citing records: A wide-scale study. Scientometrics 87:7584.

    • Search Google Scholar
    • Export Citation
  • Campbell, P 2008 Escape from the impact factor. Ethics in Science and Environmental Politics 8:57.

  • Cartwright, VA, McGhee, CNJ 2005 Ophthalmology and vision science research. Part 1: Understanding and using journal impact factors and citation indices. Journal of Cataract and Refractive Surgery 31:19992007.

    • Search Google Scholar
    • Export Citation
  • Cashore, B CG van Kooten Vertinsky, I, Auld, G, Affolderbach, J 2005 Private or self-regulation? A comparative study of forest certification choices in Canada, the United States and Germany. Forest Policy and Economics 7:5369.

    • Search Google Scholar
    • Export Citation
  • Chalmers, I 2006 Role of systematic reviews in detecting plagiarism: Case of Asim Kurjak. BMJ 333:594.

  • Chapman, S, Ragg, M, McGeechan, K 2009 Citation bias in reported smoking prevalence in people with schizophrenia. Australian and New Zealand Journal of Psychiatry 43 3 277282.

    • Search Google Scholar
    • Export Citation
  • Cheek, J, Garnham, B, Quan, J 2006 What's in a number? Issues in providing evidence of impact and quality of research(ers). Qualitative Health Research 16:423435.

    • Search Google Scholar
    • Export Citation
  • Chen, C 2006 CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American Society for Information Science and Technology 57 3 359377.

    • Search Google Scholar
    • Export Citation
  • Chew, M, Villanueva, EV MB Van Der Weyden 2007 Life and times of the impact factor: Retrospective analysis of trends for seven medical journals (1994–2005) and their Editors’ views. Journal of the Royal Society of Medicine 100:142150.

    • Search Google Scholar
    • Export Citation
  • Cobo, MJ, Lopez-Herrera, AG, Herrera-Viedma, E, Herrera, F 2011 Science mapping software tools: Review, analysis, and cooperative study among tools. JASIST 62 7 13821402.

    • Search Google Scholar
    • Export Citation
  • Cohen, JB 1938 The misuse of statistics. Journal of the American Statistical Association 33 204 657674.

  • Coleman, R 1999 Impact factors: Use and abuse in biomedical research. Anatomical Record 257:5457.

  • Coleman, A 2007 Assessing the value of a journal beyond the impact factor. Journal of the American Society for Information Science and Technology 58 8 11481161.

    • Search Google Scholar
    • Export Citation
  • Davies, J 1955 Phrenology: Fad and science: A 19th-century American crusade Yale University Press New Haven 203.

  • Dempsey, JA 2009 Impact factor and its role in academic promotion: a statement adopted by the international respiratory journal editors roundtable. Journal of Applied Physiology 107:1005.

    • Search Google Scholar
    • Export Citation
  • Dong, P, Loh, M, Mondry, A 2005 The “impact factor” revisited. Biomedical Digital Libraries 2:7.

  • Dost, FN 2008 Peer review at a crossroads: A case study. Environmental Science and Pollution Research 15:443447.

  • Drew, DE, Karpf, R 1981 Ranking academic departments: Empirical findings and a theoretical perspective. Research in Higher Education 14 4 305320.

    • Search Google Scholar
    • Export Citation
  • Eaton, L 2005 Medical editors issue guidance on ghost writing. BMJ 330 7498 988.

  • Elsevier (2011). Article posting policies. Elsevier BV. http://www.elsevier.com/wps/find/authorsview.authors/postingpolicy. Accessed 2011 September 10.

    • Search Google Scholar
    • Export Citation
  • Epstein, RJ 2004 Journal impact factors do not equitably reflect academic staff performance in different medical subspecialties. Journal of Investigative Medicine 52:531536.

    • Search Google Scholar
    • Export Citation
  • Errami, M, Garner, H 2008 A tale of two citations. Nature 451:397399.

  • Errami, M, Hicks, JM, Fisher, W, Trusty, D, Wren, JD, Long, TC, Garner, HR 2008 Deja vu: A study of duplicate citations in medline. Bioinformatics 24:243249.

    • Search Google Scholar
    • Export Citation
  • Falagas, ME, Alexiou, VG 2008 The top-ten in journal impact factor manipulation. Archivum Immunologiae et therapiae Experimentalis 56 4 223226.

    • Search Google Scholar
    • Export Citation
  • Fassoulaki, A, Paraskeva, A, Papilas, A, Karabinis, G 2000 Self-citations in six anaesthesia journals and their significance in determining the impact factor. British Journal of Anaesthesia 84 2 266269.

    • Search Google Scholar
    • Export Citation
  • Favaloro, EJ 2009 The journal impact factor: Don't expect its demise any time soon. Clinical Chemistry and Laboratory Medicine 47:13191324.

    • Search Google Scholar
    • Export Citation
  • Foo, JYA 2009 The retrospective analysis of bibliographical trends for nine biomedical engineering journals from 1999 to 2007. Annals of Biomedical Engineering 37:14741481.

    • Search Google Scholar
    • Export Citation
  • Foo J. Y. A. (2010). A retrospective analysis of the trend of retracted publications in the field of biomedical and life sciences. Science and Engineering Ethics. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Foo, JYA 2011 Impact of excessive journal self-citations: A case study on the folia phoniatrica et logopaedica journal. Science and Engineering Ethics 17:6573.

    • Search Google Scholar
    • Export Citation
  • Fox, MF 1994 Scientific misconduct and editorial and peer review processes. J Higher Educ 65:298309.

  • Frank, M 2003 Impact factors: arbiter of excellence?. J Med Libr Assoc 91:46.

  • Garfield, E 1972 Citation analysis as a tool in journal evaluation. Science 178:471479.

  • Garfield, E. (1994). The Thomson Reuters impact factor. http://thomsonreuters.com/products_services/science/free/essays/impact_factor/.

    • Search Google Scholar
    • Export Citation
  • Garfield, E 1996 How can impact factors be improved?. British Medical Journal 313:411413.

  • Garfield, E 1999 Journal impact factor: A brief review. Canadian Medical Association Journal 161:979980.

  • Garfield, E 2006 The history and meaning of the journal impact factor. JAMA 295:9093.

  • Glänzel, W 2009 The multi-dimensionality of journal impact. Scientometrics 78:355374.

  • Glänzel, W, Moed, HF 2002 Journal impact measures in bibliometric research. Scientometrics 53:171193.

  • Gluud, LL, Sorensen, TIA, Gotzsche, PC, Gluud, C 2005 The journal impact factor as a predictor of trial quality and outcomes: Cohort study of hepatobiliary randomized clinical trials. American Journal of Gastroenterology 100:24312435.

    • Search Google Scholar
    • Export Citation
  • Glynn, RW, Chin, JZ, Kerin, MJ, Sweeney, KJ 2010 Representation of cancer in the medical literature: A bibliometric analysis. PLoS ONE 5 11 e13902.

    • Search Google Scholar
    • Export Citation
  • Godlee, F 2004 Dealing with editorial misconduct. BMJ 329:1301.

  • Goldstein, H, Maier, G 2010 The use and valuation of journals in planning scholarship: Peer assessment versus impact factors. Journal of Planning Education and Research 30:6675.

    • Search Google Scholar
    • Export Citation
  • Gollogly, L, Momen, H 2006 Ethical dilemmas in scientific publication: pitfalls and solutions for editors. Rev Saúde Pública 40:2429.

    • Search Google Scholar
    • Export Citation
  • Gøtzsche, PC, Kassirer, JP, Woolley, KL, Wager, E, Jacobs, A et al. 2009 What should be done to tackle ghost writing in the medical literature?. PLoS Med 6 2 e1000023 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Greenwood, DC 2007 Reliability of journal impact factor rankings. BMC Medical Research Methodology 7:48.

  • Gwilym, SE, Swan, MC, Giele, H 2004 One in 13 ‘original’ articles in the journal of bone and joint surgery are duplicate or fragmented publications. The Journal of Bone and Joint Surgery 86-B 5 743745.

    • Search Google Scholar
    • Export Citation
  • Ha, TC, Tan, SB, Soo, KC 2006 The journal impact factor: Too much of an impact?. Annals of the Academy of Medicine, Singapore 35:911916.

    • Search Google Scholar
    • Export Citation
  • Habibzadeh, F, Winker, MA 2009 Duplicate publication and plagiarism: causes and cures. Notfall Rettungsmed 12:415418.

  • Hames, I 2007 Peer review and manuscript management in scientific journals: guidelines for good practice Wiley Oxford .

  • Hansen, HB, Henriksen, JH 1997 How well does journal ‘impact’ work in the assessment of papers on clinical physiology and nuclear medicine?. Clinical Physiology 17 4 409418.

    • Search Google Scholar
    • Export Citation
  • Haslam, N, Koval, P 2010 Predicting long-term citation impact of articles in social and personality psychology. Psychological Reports 106 3 891900.

    • Search Google Scholar
    • Export Citation
  • Hernan, MA 2009 Impact factor: A call to reason. Epidemiology 20:317318.

  • Ho, D 2011 Thomson Reuters: Cause for celebration. Journal of Laboratory Automation 16 3 A7A8.

  • Holsapple, CW 2009 Journal self-citation II: The quest for high impact: Truth and consequences?. Communications of the Association for Information Systems 25:1120.

    • Search Google Scholar
    • Export Citation
  • Horner, J, Minifie, FD 2011 Research ethics III: Publication practices and authorship, conflicts of interest, and research misconduct. Journal of Speech, Language, and Hearing S346 Research 54:S346S362.

    • Search Google Scholar
    • Export Citation
  • Hunt, GE, Cleary, M, Walter, G 2010 Psychiatry and the Hirsch h-index: The relationship between journal impact factors and accrued citations. Harvard Review of Psychiatry 18 4 207219.

    • Search Google Scholar
    • Export Citation
  • Jacobs, G, Ip, B 2005 Ring fenced research: The case of computer-assisted learning in health sciences. British Journal of Educational Technology 36:361377.

    • Search Google Scholar
    • Export Citation
  • Jacso, P 2009 Five-year impact factor data in the journal citation reports. Online Information Review 33:603614.

  • Jahangiriana, M, Eldabi, T, Garg, L, Jun, GT, Naseer, A, Patel, B, Stergioulas, L, Young, T 2011 A rapid review method for extremely large corpora of literature: Applications to the domains of modelling, simulation, and management. International Journal of Information Management 31:234243.

    • Search Google Scholar
    • Export Citation
  • Jeacle, I., & Carter, C. (2011). In TripAdvisor we trust: Rankings, calculative regimes and abstract systems. Accounting, Organizations and Society. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Johnson, C 2006 Repetitive, duplicate, and redundant publications: a review for authors and readers. Journal of Manipulative and Physiological Therapeutics 29:505509.

    • Search Google Scholar
    • Export Citation
  • Johnstone, M-J 2007 Journal impact factors: Implications for the nursing profession. International Nursing Review 54:3540.

  • Jones, AW 2002 JAT's impact factor: Room for improvement?. Journal of Analytical Toxicology 26:25.

  • Jones, AW 2003 Impact factors of forensic science and toxicology journals: What do the numbers really mean?. Forensic Science International 133:18.

    • Search Google Scholar
    • Export Citation
  • Jörgensen, P 2005 Incorporating context in text analysis by interactive activation with competition artificial neural networks. Information Processing & Management 41 5 10811099.

    • Search Google Scholar
    • Export Citation
  • Kapeller, J 2010 Citation metrics: Serious drawbacks, perverse incentives, and strategic options for heterodox economics. American Journal of Economics and Sociology 69:13761408.

    • Search Google Scholar
    • Export Citation
  • Ketcham, CM 2008 The proper use of citation data in journal management. Archivum Immunologiae et therapiae Experimentalis 56:357362.

  • Knothe, G 2006 Comparative citation analysis of duplicate or highly related publications. Journal of the American Society for Information Science and Technology 57 13 18301839.

    • Search Google Scholar
    • Export Citation
  • Kotiaho, JS, Tomkins, JL, Simmons, LW 1999 Unfamiliar citations breed mistakes. Nature 400:307.

  • Krell, F-T 2010 Should editors influence journal impact factors?. Learned Publishing 23:5962.

  • Kumar, V, Upadhyay, S, Medhi, B 2009 Impact of the impact factor in biomedical research: Its use and misuse. Singapore Medical Journal 50:752755.

    • Search Google Scholar
    • Export Citation
  • Kurmis, AP 2003 Understanding the limitations of the journal impact factor. Journal of Bone and Joint Surgery: Series A 85:24492454.

  • Kurmis, AP, Kurmis, TP 2006 Exploring the relationship between impact factor and manuscript rejection rates in radiologic journals. Academic Radiology 13:7783.

    • Search Google Scholar
    • Export Citation
  • Kurmis, TP, Kurmis, AP 2010 Self-citation rates among medical imaging journals and a possible association with impact factor. Radiography 16:2125.

    • Search Google Scholar
    • Export Citation
  • Laband, DN 1990 Is there value-added from the review process in economics?: Preliminary evidence from authors. The Quarterly Journal of Economics 105:341352.

    • Search Google Scholar
    • Export Citation
  • Lange, LL 2002 The impact factor as a phantom: Is there a self-fulfilling prophecy effect of impact?. Journal of Documentation 58:175184.

    • Search Google Scholar
    • Export Citation
  • Larivière, V, Gingras, Y 2010 The impact factor's Matthew effect: A natural experiment in bibliometrics. Journal of the American Society for Information Science and Technology 61:424427.

    • Search Google Scholar
    • Export Citation
  • Lau, SL, Samman, N 2007 Levels of evidence and journal impact factor in oral and maxillofacial surgery. International Journal of Oral and Maxillofacial Surgery 36:15.

    • Search Google Scholar
    • Export Citation
  • Lawrence, PA 2007 The mismeasurement of science. Current Biology 17:R583R585.

  • Lehmkuhl, G, Petermann, F, Warnke, A 2009 Research in child and adolescent psychiatry, promotion of young academics and publication practice [Kinder- und jugendpsychiatrische Forschung, Förderung des wissenschaftlichen Nachwuchses und Veröffentlichungspraxis]. Zeitschrift fur Kinder- und Jugendpsychiatrie und Psychotherapie 37:9396.

    • Search Google Scholar
    • Export Citation
  • Leutner, D, Wirth, J 2007 As mirrored by the journal: Themes and trends of educational psychology in the years 2005 to 2007 [Im Spiegel der Zeitschrift: Themen und Trends der Pädagogischen Psychologie in den Jahren 2005 bis 2007]. Zeitschrift fur Padagogische Psychologie 21:195202.

    • Search Google Scholar
    • Export Citation
  • Leydesdorff, L, Opthof, T 2010 Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology 61:23652369.

    • Search Google Scholar
    • Export Citation
  • Lock, S 1995 Lessons from the Pearce affair: Handling scientific fraud. BMJ 310:15471548.

  • Lomnicki, A 2003 Impact factors reward and promote excellence. Nature 424:487.

  • Maier, G 2006 Impact factors and peer judgment: The case of regional science journals. Scientometrics 69 3 651667.

  • McGarty, C 2000 The citation impact factor in social psychology: a bad statistic that encourages bad science. Current Research in Social Psychology 5:116.

    • Search Google Scholar
    • Export Citation
  • McKeever, L 2006 Online plagiarism detection services: Saviour or scourge?. Assessment & Evaluation in Higher Education 31:155165.

  • McVeigh, M. E. (2004). Journal self-citation in the journal citation reports. Thomson Reuters, http://thomsonreuters.com/products_services/science/free/essays/journal_self_citation_jcr. Accessed 2011 September 7.

    • Search Google Scholar
    • Export Citation
  • McVeigh, ME, Mann, SJ 2009 The journal impact factor denominator: Defining citable (counted) items. JAMA 302:11071109.

  • Medarević, M., Banković, S., Pantić, D. & Obradović, S. (2010) Effects of the control method (goč variety) in selection forest management in western Serbia. Archives of Biological Science, 62 (2), 407418.

    • Search Google Scholar
    • Export Citation
  • Mehrad, J, Goltaji, M 2010 Correlation between journal self citation with impact factor for the scientific publications in humanities published between 2001 and 2007 based on Persian journal citation report generated by Islamic science citation database. Information Sciences and Technology 25:189206.

    • Search Google Scholar
    • Export Citation
  • Mehrad, J, Goltaji, M 2011 Correlation between journal self-citation and impact factor in ISC's PJCR agriculture and veterinary science journals during 2001–2007. International Journal of Information Science and Management 9:7587.

    • Search Google Scholar
    • Export Citation
  • Meneghini, R, Packer, AL, Nassi-Calo, L 2008 Articles by Latin American authors in prestigious journals have fewer citations. PLoS ONE 3 11 e3804.

    • Search Google Scholar
    • Export Citation
  • Metze, K 2010 Bureaucrats, researchers, editors and the impact factor: A vicious circle that is detrimental to science. Clinics 65 10 937940.

    • Search Google Scholar
    • Export Citation
  • Meyer zu Eissen, S, Stein, B 2006 Intrinsic plagiarism detection. Lecture Notes in Computer Science 3936:565569.

  • Minelli, C, Thompson, JR, Abrams, KR, Thakkinstian, A, Attia, J 2009 The quality of meta-analyses of genetic association studies: A review with recommendations. American Journal of Epidemiology 170:13331343.

    • Search Google Scholar
    • Export Citation
  • Moed, H, Plume, A 2011 Research assessment: The multi-dimensional research assessment matrix. Research Trends 23:57.

  • Moed, HF Th N Van Leeuwen Reedijk, J 1996 A critical analysis of the journal impact factors of Angewandte chemie and the Journal of the American Chemical Society: Inaccuracies in published impact factors based on overall citations only. Scientometrics 37:105116.

    • Search Google Scholar
    • Export Citation
  • Moller, J, Retelsdorf, J, Sudkamp, A 2010 As mirrored by the journal: Themes and trends of educational psychology in the years 2008 to 2010 [Im Spiegel der Zeitschrift: Themen und Trends der Pädagogischen Psychologie in den Jahren 2008 bis 2010]. Zeitschrift fur Padagogische Psychologie 24:163169.

    • Search Google Scholar
    • Export Citation
  • Monastersky, R. (2005). The number that's devouring science. The chronicle of higher education 52 (8): A12 http://chronicle.com/free/v52/i08/08a01201.htm.

    • Search Google Scholar
    • Export Citation
  • Mullen, L. B. (2008). Increasing Impact of Scholarly Journal Articles: Practical Strategies Librarians Can Share. E-JASL: The Electronic Journal of Academic and Special Librarianship 9, 1, http://southernlibrarianship.icaap.org/content/v09n01/mullen_l01.html. Accessed 2011 September 1.

    • Search Google Scholar
    • Export Citation
  • Neuhaus, C, Marx, W, Daniel, H-D 2009 The publication and citation impact profiles of Angewandte Chemie and the journal of the American Chemical Society based on the sections of chemical abstracts: A case study on the limitations of the journal impact factor. Journal of the American Society for Information Science and Technology 60:176183.

    • Search Google Scholar
    • Export Citation
  • Obremskey, WT, Pappas, N, Attallah-Wasif, E P Tornetta III Bhandari, M 2005 Level of evidence in orthopaedic journals. Journal of Bone and Joint Surgery: Series A 87 12 I 26322638.

    • Search Google Scholar
    • Export Citation
  • Ogden, TL, Bartley, DL 2008 The ups and downs of journal impact factors. Annals of Occupational Hygiene 52 2 7382.

  • Owlia, P, Vasei, M, Goliaei, B, Nassiri, I 2011 Normalized impact factor (NIF): An adjusted method for calculating the citation rate of biomedical journals. Journal of Biomedical Informatics 44:216220.

    • Search Google Scholar
    • Export Citation
  • Patterson, M 2009 Is the end in cite?. EMBO reports 10 11 1186.

  • Perneger, TV 2010 Citation analysis of identical consensus statements revealed journal-related bias. Journal of Clinical Epidemiology 63 6 660664.

    • Search Google Scholar
    • Export Citation
  • PLoS Medicine Editors 2006 The impact factor game. PLoS Med 3 6 e291 .

  • Poomkottayil, D, Bornstein, MM, Sendi, P 2011 Lost in translation: The impact of publication language on citation frequency in the scientific dental literature. Swiss Medical Weekly 141:w13148.

    • Search Google Scholar
    • Export Citation
  • Porter, M, McIntyre, S 1984 What is, must be best: A research note on conservative or deferential responses to antenatal care provision. Social Science and Medicine 19:11971200.

    • Search Google Scholar
    • Export Citation
  • Postma, E 2007 Inflated impact factors? The true impact of evolutionary papers in non-evolutionary journals. PLoS ONE 2 10 e999.

  • Preti, F, Dani, A, Laio, F 2010 Root profile assessment by means of hydrological, pedological and above-ground vegetation information for bio-engineering purposes. Ecological Engineering 36:305316.

    • Search Google Scholar
    • Export Citation
  • Pudovkin, AI, Garfield, E 2004 Rank-normalized impact factor: A way to compare journal performance across subject categories. Proceedings of the ASIST Annual Meeting 41:507515.

    • Search Google Scholar
    • Export Citation
  • Racki, G 2009 Rank-normalized journal impact factor as a predictive tool. Archivum Immunologiae et therapiae Experimentalis 57:3943.

  • Ramsden, JJ 2009 Impact factors: A critique. Journal of Biological Physics and Chemistry 9:139140.

  • Reedijk, J, Moed, HF 2008 Is the impact of journal impact factors decreasing?. Journal of Documentation 64:183192.

  • Reller, T. (2011). Elsevier announces 2010 journal impact factor highlights. Press release, 14 July 2011. http://www.marketwatch.com/story/elsevier-announces-2010-journal-impact-factor-highlights-2011-07-14. Accessed 2011 August 25.

    • Search Google Scholar
    • Export Citation
  • Resnik, DB, Peddada, S W Brunson Jr 2009 Research misconduct policies of scientific journals. Accountability in Research 16:254267.

  • Resnik, DB, Patrone, D, Peddada, S 2010 Research misconduct policies of social science journals and impact factor. Accountability in Research 17:7984.

    • Search Google Scholar
    • Export Citation
  • Rezaei-Ghaleh, N, Azizi, F 2007 The impact factor-based quality assessment of biomedical research institutes in Iran: Effect of impact factor normalization by subject. Archives of Iranian Medicine 10:182189.

    • Search Google Scholar
    • Export Citation
  • Rieseberg, L, Smith, H 2008 Editorial and Retrospective. Molecular Ecology 17:501513.

  • Rieseberg, L, Vines, T, Kane, N 2011 20 years of molecular ecology. Molecular Ecology 20:121.

  • Robert, CP 2010 On the relevance of the Bayesian approach to statistics. Review of Economic Analysis 2:139152.

  • Roberts, J 2009 An author's guide to publication ethics: A review of emerging standards in biomedical journals. Headache 49:578589.

  • Roosendaal, H. E., Geurts, P. A. T. M. (1997). Forces and functions in scientific communication: An analysis of their interplay. Cooperative Research Information Systems in Physics, August 31-September 4 1997, Oldenburg. http://www.physik.unioldenburg.de/conferences/crisp97/roosendaal.html.

    • Search Google Scholar
    • Export Citation
  • Rossner, M 2006 How to guard against image fraud. The Scientist 20 3 2425.

  • Rossner, M H Van Epps Hill, E 2007 Show me the data. Journal of Cell Biology 179:10911092.

  • Rousseau, R 2002 Journal evaluation: Technical and practical issues. Library Trends 50:418.

  • Rousseau, R G Van Hooydonk 1996 Journal production and journal impact factors. Journal of the American Society for Information Science 47:775780.

    • Search Google Scholar
    • Export Citation
  • Ruiz, MA, Greco, OT, Braile, DM 2009 Journal impact factor: This editorial, academic and scientific influence [Fator de impacto: Importancia e influencia no meio editorial, academico e cientifico]. Brazilian Journal of Cardiovascular Surgery 24:273278.

    • Search Google Scholar
    • Export Citation
  • Saha, S, Saint, S, Christakis, DA 2003 Impact factor: A valid measure of journal quality?. Journal of the Medical Library Association 91:4246.

    • Search Google Scholar
    • Export Citation
  • Sample, I. (2002). People weigh less on a hard surface. New Scientist http://www.newscientist.com/article/dn2462-people-weigh-less-on-a-hard-surface.html.

    • Search Google Scholar
    • Export Citation
  • Saunders, R, Savulescu, J 2008 Research ethics and lessons from Hwanggate: what can we learn from the Korean cloning fraud?. Journal of Medical Ethics 34:214221.

    • Search Google Scholar
    • Export Citation
  • Schoonbaert, D, Roelants, G 1996 Citation analysis for measuring the value of scientific publications: Quality assessment tool or comedy of errors?. Tropical Medicine and International Health 1:739752.

    • Search Google Scholar
    • Export Citation
  • Schopfel, J, Prost, H 2009 Comparison of SCImago journal rank indicator (SJR) with JCR journal impact factor (IF) for French journals [Le JCR facteur d'impact (IF) et le SCImago journal rank indicator (SJR) des revues françaises: une étude comparative]. Psychologie Francaise 54:287305.

    • Search Google Scholar
    • Export Citation
  • Schubert, A, Glanzel, W 1983 Statistical reliability of comparisons based on the citation impact of scientific publications. Scientometrics 5:5973.

    • Search Google Scholar
    • Export Citation
  • Schumm, WR 2010 A comparison of citations across multidisciplinary psychology journals: A case study of two independent journals. Psychological Reports 106:314322.

    • Search Google Scholar
    • Export Citation
  • Scully, C, Lodge, H 2005 Impact factors and their significance; overrated or misused?. British Dental Journal 198:391393.

  • Seglen, PO 1989 Use of citation analysis and other bibliometric methods in evaluation of the quality of research [Bruk av siteringsanalyse og andre bibliometriske metoder i evaluering av forskningskvalitet]. Tidsskrift for den Norske Laegeforening 109 31 32243229.

    • Search Google Scholar
    • Export Citation
  • Seglen, PO 1992 Journal impact: How representative is the journal impact factor?. Research Evaluation 2:143149.

  • Seglen, PO 1997 Why the impact factor of journals should not be used for evaluating research. BMJ 314:497502.

  • Simkin, MV, Roychowdhury, VP 2003 Read before you cite. Complex Syst 14:269274.

  • Simons, K 2008 The misused impact factor. Science 322 5899 165.

  • Skovsgaard, JP, Vanclay, JK 2008 Forest site productivity: A review of the evolution of dendrometric concepts for even-aged stands. Forestry 81:1331.

    • Search Google Scholar
    • Export Citation
  • Smith, DR 2007 Historical development of the journal impact factor and its relevance for occupational health. Industrial Health 45:730742.

    • Search Google Scholar
    • Export Citation
  • Sombatsompop, N, Markpin, T 2005 Making an equality of ISI impact factors for different subject fields. Journal of the American Society for Information Science and Technology 56:676683.

    • Search Google Scholar
    • Export Citation
  • Sombatsompop, N, Markpin, T, Premkamolnetr, N 2004 A modified method for calculating the impact factors of journals in ISI journal citation reports: Polymer science category in 1997–2001. Scientometrics 60:217235.

    • Search Google Scholar
    • Export Citation
  • Sonderstrup-Andersen, EM, Sonderstrup-Andersen, HHK 2008 An investigation into diabetes researcher's perceptions of the journal impact factor: Reconsidering evaluating research. Scientometrics 76:391406.

    • Search Google Scholar
    • Export Citation
  • Soreide, K, Winter, DC 2010 Global survey of factors influencing choice of surgical journal for manuscript submission. Surgery 147:475480.

    • Search Google Scholar
    • Export Citation
  • Starbuck, WH 2005 How much better are the most-prestigious journals? The statistics of academic publication. Organization Science 16:180200.

    • Search Google Scholar
    • Export Citation
  • Statzner, B, Resh, VH 2010 Negative changes in the scientific publication process in ecology: Potential causes and consequences. Freshwater Biology 55:26392653.

    • Search Google Scholar
    • Export Citation
  • Stein, RJ, Haddock, CK, Poston, WSC, Catanese, D, Spertus, JA 2005 Precision in weighing: A comparison of scales found in physician offices, fitness centers, and weight loss centers. Public Health Reports 120:266270.

    • Search Google Scholar
    • Export Citation
  • Stiftel, B, Mukhopadhyay, C 2007 Thoughts on Anglo-American hegemony in planning scholarship: Do we read each other's work?. Town Planning Review 78:545572.

    • Search Google Scholar
    • Export Citation
  • Straub, DW, Anderson, C 2009 Journal self-citation VI: Forced journal self-citation: Common, appropriate, ethical?. Communications of the Association for Information Systems 25:5766.

    • Search Google Scholar
    • Export Citation
  • Tal, A, Gordon, J 2010 Carbon cautious: Israel's afforestation experience and approach to sequestration. Small-Scale Forestry 9:409428.

    • Search Google Scholar
    • Export Citation
  • Taylor, M, Perakakis, P, Trachana, V 2008 The siege of science. Ethics in Science and Environmental Politics 8:1740.

  • Thomson Reuters (2011). Journal citation reports. Thomson reuters products and services. http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/. Accessed 2011 August 16.

    • Search Google Scholar
    • Export Citation
  • Timuralp, B 2010 Our first impact factor. Anadolu Kardiyoloji Dergisi 10 4 297.

  • Tobin, MJ 2002 AJRCCM's policy on duplicate publication. American Journal of Respiratory and Critical Care Medicine 166:433437.

  • Todd, PA, Ladle, RJ 2008 Hidden dangers of a ‘citation culture’. Ethics in Science and Environmental Politics 8:1316.

  • Todd, PA, Guest, JR, Lu, J, Chou, LM 2010 One in four citations in marine biology papers is inappropriate. Marine Ecology Progress Series 408:299303.

    • Search Google Scholar
    • Export Citation
  • Ugolini, D, Bogliolo, A, Parodi, S, Casilli, C, Santi, L 1997 Assessing research productivity in an oncology research institute: The role of the documentation center. Bulletin of the Medical Library Association 85:3338.

    • Search Google Scholar
    • Export Citation
  • H Van de Sompel Payette, S, Erickson, J, Lagoze, C, Warner, S 2004 Rethinking scholarly communication: Building the system that scholars deserve. D-Lib Magazine 10:9.

    • Search Google Scholar
    • Export Citation
  • M Van Driel J De Maeseneer A De Sutter D De Bacquer G De Backer Christiaens, T 2008 How scientific is the assessment of the quality of scientific output using the journal impact factor? [Hoe wetenschappelijk is het beoordelen van wetenschappelijk werk aan de hand van impactfactoren van tijdschriften?]. Tijdschrift voor Geneeskunde 64:471476.

    • Search Google Scholar
    • Export Citation
  • Van Gyseghem, E., Baert, L., Van Remoortere, P., van‘t Klooster, G., Rouan, M.-C., Voorspoels, J., de Kock, H., Schueller, L., Rosier, J., Grooten, L., & Van den Mooter, G. (2010) Co-administration of darunavir and a new pharmacokinetic booster: Formulation strategies and evaluation in dogs. European Journal of Pharmaceutical Sciences, 41, 193200.

    • Search Google Scholar
    • Export Citation
  • TN Van Leeuwen Moed, HF, Reedijk, J 1999 Critical comments on institute for scientific information impact factors: a sample of inorganic molecular chemistry journals. Journal of Information Science 25 6 189198.

    • Search Google Scholar
    • Export Citation
  • Vanclay, JK 2009 Bias in the journal impact factor. Scientometrics 78:312.

  • Vlosky, RP, Ozanne, LK 1997 Forest products certification: The business customer perspective. Wood and Fiber Science 29:195208.

  • Wager, E, Fiack, S, Graf, C, Robinson, A, Rowlands, I 2009 Science journal editors’ views on publication ethics: Results of an international survey. Journal of Medical Ethics 35:348353.

    • Search Google Scholar
    • Export Citation
  • Wagner, A. B. (2009). Percentile-based journal impact factors: A neglected collection development metric. Issues in Science and Technology Librarianship 57.

    • Search Google Scholar
    • Export Citation
  • Walter, G, Bloch, S, Hunt, G, Fisher, K 2003 Counting on citations: A flawed way to measure quality. Medical Journal of Australia 178 6 280281.

    • Search Google Scholar
    • Export Citation
  • Wang, S, Wang, H, Weldon, PR 2010 Effect of cooperation between Chinese scientific journals and international publishers on journals’ impact factor. Journal of Informetrics 4:233238.

    • Search Google Scholar
    • Export Citation
  • Weale, AR, Bailey, M, Lear, PA 2004 The level of non-citation of articles within a journal as a measure of quality: A comparison to the impact factor. BMC Medical Research Methodology 4:14.

    • Search Google Scholar
    • Export Citation
  • Weller, A. C. (2001). Editorial peer review: Its strengths and weaknesses. American society for information science and technology monograph series, Information Today, Inc (p. 342). ISBN 9781573871006.

    • Search Google Scholar
    • Export Citation
  • Williams, G 2007 Should we ditch impact factors?. BMJ 334:568.

  • Winkmann, G, Schweim, HG 2000 Biomedical databases and the journal impact factor [Medizinisch-biowissenschaftliche datenbanken und der impact-faktor]. Deutsche Medizinische Wochenschrift 125:11331141.

    • Search Google Scholar
    • Export Citation
  • Winkmann, G, Schlutius, S, Schweim, HG 2002 Publication languages of Impact factor journals and of medical bibliographic databanks [Publikationssprachen der Impact Faktor-zeitschriften und medizinischer literaturdatenbanken]. Deutsche Medizinische Wochenschrift 127:131137.

    • Search Google Scholar
    • Export Citation
  • Woelfel, J 1993 Artificial neural networks in policy research: A current assessment. Journal of Communication 43:6380.

  • Woelfel, J., Woelfel, J. (1997). ThoughtView version 2.0, Galileo Corporation.

  • Woolgar, S 1991 Beyond the citation debate: towards a sociology of measurement technologies and their use in science policy. Science and Public Policy 18:319326.

    • Search Google Scholar
    • Export Citation
  • Wulff, JL, Nixon, ND 2004 Quality markers and use of electronic journals in an academic health sciences library. Journal of the Medical Library Association 92 3 315322.

    • Search Google Scholar
    • Export Citation
  • Xiao, H, Yuan, F, Wu, J-G 2009 Factors affecting citations: A comparison between Chinese and English journals in ecology. Chinese Journal of Applied Ecology 20:12531262.

    • Search Google Scholar
    • Export Citation
  • Yu, G, Wang, L 2007 The self-cited rate of scientific journals and the manipulation of their impact factors. Scientometrics 73:321330.

  • Yu, G, Yang, D-H, He, H-X 2011 An automatic recognition method of journal impact factor manipulation. Journal of Information Science 37:235245.

    • Search Google Scholar
    • Export Citation
  • Yue, W, Wilson, CS, Boller, F 2007 Peer assessment of journal quality in clinical neurology. Journal of the Medical Library Association 95:7076.

    • Search Google Scholar
    • Export Citation
  • Zetterstrom, R 1999 Impact factor and the future of Acta Paediatrica and other European medical journals. Acta Paediatrica, International Journal of Paediatrics 88:793796.

    • Search Google Scholar
    • Export Citation