Authors:
Lutz Bornmann Max Planck Society, Hofgartenstr 8, 80539 Munich, Germany

Search for other papers by Lutz Bornmann in
Current site
Google Scholar
PubMed
Close
,
Hermann Schier Max Planck Institute for Solid State Research, Heisenbergstraße 1, 70569 Stuttgart, Germany

Search for other papers by Hermann Schier in
Current site
Google Scholar
PubMed
Close
,
Werner Marx Max Planck Institute for Solid State Research, Heisenbergstraße 1, 70569 Stuttgart, Germany

Search for other papers by Werner Marx in
Current site
Google Scholar
PubMed
Close
, and
Hans-Dieter Daniel ETH Zurich, Professorship for Social Psychology and Research on Higher Education, Zähringerstr 24, 8092 Zurich, Switzerland
University of Zurich, Evaluation Office, Mühlegasse 21, 8001 Zurich, Switzerland

Search for other papers by Hans-Dieter Daniel in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Schubert (Scientometrics, 78:559–565, 2009) showed that “a Hirsch-type index can be used for assessing single highly cited publications by calculating the h index of the set of papers citing the work in question” (p. 559). To demonstrate that this single publication h index is a useful yardstick to compare the quality of different publications; the index should be strongly related to the assessment by peers. In a comprehensive research project we investigated the peer review process of the Angewandte Chemie International Edition. The data set contains manuscripts reviewed in the year 2000 and accepted by the journal or rejected but published elsewhere. Single publication h index values were calculated for a total of 1,814 manuscripts. The results show a correlation in the expected direction between peer assessments and single publication h index values: After publication, manuscripts with positive ratings by the journal's reviewers show on average higher h index values than manuscripts with negative ratings by reviewers (and later published elsewhere). However, our findings do not support Schubert's (2009) assumption that the additional dimension of indirect citation influence contributes to a more refined picture of the most cited papers.

  • Bornmann, L 2011 Scientific peer review. Annual Review of Information Science and Technology 45:199245.

  • Bornmann, L, Daniel, H-D 2007 What do we know about the h index?. Journal of the American Society for Information Science and Technology 58 9 13811385 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Daniel, H-D 2008 The effectiveness of the peer review process: Inter-referee agreement and predictive validity of manuscript refereeing at Angewandte Chemie. Angewandte Chemie International Edition 47 38 71737178 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Daniel, H-D 2008 Selecting manuscripts for a high impact journal through peer review: A citation analysis of Communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology 59 11 18411852 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Daniel, H-D 2009 Extent of type I and type II errors in editorial decisions: A case study on Angewandte Chemie International Edition. Journal of Informetrics 3 4 348352 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Daniel, H-D 2009 The luck of the referee draw: The effect of exchanging reviews. Learned Publishing 22 2 117125 .

  • Bornmann, L, Daniel, H-D 2009 The state of h index research. Is the h index the ideal way to measure research performance?. EMBO Reports 10 1 26 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Daniel, H-D 2010 The manuscript reviewing process: Empirical research on review requests, review sequences and decision rules in peer review. Library and Information Science Research 32 1 512 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Marx, W, Schier, H, Rahm, E, Thor, A, Daniel, HD 2009 Convergent validity of bibliometric Google Scholar data in the field of chemistry. Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts. Journal of Informetrics 3 1 2735 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Mutz, R, Daniel, H-D 2007 The b index as a measure of scientific excellence. A promising supplement to the h index. Cybermetrics 11 1 paper 6.

    • Search Google Scholar
    • Export Citation
  • Bornmann, L., Mutz, R., Marx, W., Schier, H., & Daniel, H.-D. (2011). A multilevel modelling approach to investigating the predictive validity of editorial decisions: Do the editors of a high-profile journal select manuscripts that are highly cited after publication? Journal of the Royal Statistical Society-Series A (Statistics in Society), 174 (4). doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Weymuth, C, Daniel, H-D 2010 A content analysis of referees’ comments: How do comments on manuscripts rejected by a high-impact journal and later published in either a low- or high-impact journal differ?. Scientometrics 83 2 493506 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cole, S 1989 Citations and the evaluation of individual scientists. Trends in Biochemical Sciences 14 1 913 .

  • Franceschini, F, Maisano, DA 2010 Analysis of the Hirsch index's operational properties. European Journal of Operational Research 203 2 494504 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hirsch, JE 2005 An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America 102 46 1656916572 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schubert, A 2009 Using the h-index for assessing single publications. Scientometrics 78 3 559565 .

  • Sheskin, D 2007 Handbook of parametric and nonparametric statistical procedures 4 Chapman & Hall/CRC Boca Raton.

  • StataCorp. 2009 Stata Statistical Software: Release 11 Stata Corporation College Station.

  • Thorngate, W, Dawes, RM, Foddy, M 2009 Judging merit Psychology Press New York.

  • NJ van Eck Waltman, L 2008 Generalizing the h- and g-indices. Journal of Informetrics 2 4 263271 .

  • Collapse
  • Expand

To see the editorial board, please visit the website of Springer Nature.

Manuscript submission: http://www.editorialmanager.com/scim/

For subscription options, please visit the website of Springer Nature.

Scientometrics
Language English
Size B5
Year of
Foundation
1978
Volumes
per Year
1
Issues
per Year
12
Founder Akadémiai Kiadó
Founder's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Publisher Akadémiai Kiadó
Springer Nature Switzerland AG
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
CH-6330 Cham, Switzerland Gewerbestrasse 11.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 0138-9130 (Print)
ISSN 1588-2861 (Online)