Abstract
Whereas in traditional, closed peer review (CPR) a few, selected scientists (peers) are included in the process of manuscript review, public peer review (PPR) includes, in addition to invited reviewers, a wider circle of scientists who are interested in a manuscript and wish to write a comment on it. In this study, using the data of two comprehensive evaluation studies on the CPR process at Angewandte Chemie—International Edition and the PPR process at Atmospheric Chemistry and Physics, we examined the language characteristics in comments that were written by invited reviewers in CPR and by invited reviewers and interested members of the scientific community in PPR. We used Linguistic Inquiry and Word Count (LIWC), a text analysis software program that counts words in meaningful categories (e.g., positive or negative emotions) using a standardized dictionary. We examined 599 comments from the reviews of 229 manuscripts. The results show that the comments in PPR are much longer than the comments in CPR. This is an indication that PPR reviewing has more of an improvement function and CPR reviewing has more of a selection function. The results also show that CPR is not, as might be expected, more susceptible to the expression of negative emotions than PPR is. On the contrary, positive emotion words are used statistically significantly more frequently in CPR than in PPR.
ProCite
RefWorks
Reference Manager
BibTeX
Zotero
EndNote
-
Anon. (2006). Peer review on trial. Nature, 441 (7094), 668.
-
Bakanic, V, McPhail, C, Simon, RJ 1989 Mixed messages: Referees’ comments on the manuscripts they review. Sociological Quarterly 30 4 639–654 .
-
Bingham, CM, Higgins, G, Coleman, R MB Van Der Weyden 1998 The Medical Journal of Australia Internet peer-review study. Lancet 352 9126 441–445 .
-
Bornmann, L, Daniel, H-D 2008 The effectiveness of the peer review process: Inter-referee agreement and predictive validity of manuscript refereeing at Angewandte Chemie. Angewandte Chemie International Edition 47 38 7173–7178 .
-
Bornmann, L, Daniel, H-D 2008 Selecting manuscripts for a high impact journal through peer review: A citation analysis of communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology 59 11 1841–1852 .
-
Bornmann, L, Daniel, H-D 2009 Extent of type I and type II errors in editorial decisions: A case study on Angewandte Chemie International Edition. Journal of Informetrics 3 4 348–352 .
-
Bornmann, L, Daniel, H-D 2009 Reviewer and editor biases in journal peer review: An investigation of manuscript refereeing at Angewandte Chemie International Edition. Research Evaluation 18 4 262–272 .
-
Bornmann, L, Daniel, H-D 2010 How long is the peer review process for journal manuscripts? A case study on Angewandte Chemie International Edition. Chimia 64 1–2 72–77 .
-
Bornmann, L, Daniel, H-D 2010 Reliability of reviewers’ ratings at an interactive open access journal using public peer review: A case study on Atmospheric Chemistry and Physics. Learned Publishing 23 2 124–131 .
-
Bornmann, L., & Daniel, H.-D. (2010c). The manuscript reviewing process—empirical research on review requests, review sequences and decision rules in peer review. Library & Information Science Research, 32 (1), 5–12.
-
Bornmann, L, Daniel, H-D 2010 The validity of staff editors’ initial evaluations of manuscripts: A case study of Angewandte Chemie International Edition. Scientometrics 85 3 681–687 .
-
Bornmann, L, Marx, W, Schier, H, Rahm, E, Thor, A, Daniel, HD 2009 Convergent validity of bibliometric Google Scholar data in the field of chemistry. Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts. Journal of Informetrics 3 1 27–35 .
-
Bornmann, L, Marx, W, Schier, H, Thor, A, Daniel, H-D 2010 From black box to white box at open access journals: Predictive validity of manuscript reviewing and editorial decisions at Atmospheric Chemistry and Physics. Research Evaluation 19 2 105–118 .
-
Bornmann, L, Nast, I, Daniel, H-D 2008 Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics 77 3 415–432 .
-
Bornmann, L, Neuhaus, C, Daniel, H-D 2011 The effect of a two-stage publication process on the Journal Impact Factor: A case study on the interactive open access journal Atmospheric Chemistry and Physics. Scientometrics 86:93–97 .
-
Bornmann, L, Schier, H, Marx, W, Daniel, H-D 2011 Is interactive open access publishing able to identify high-impact submissions? A study on the predictive validity of Atmospheric Chemistry and Physics by using percentile rank classes. Journal of the American Society for Information Science and Technology 62 1 61–71 .
-
Bornmann, L, Weymuth, C, Daniel, H-D 2010 A content analysis of referees’ comments: How do comments on manuscripts rejected by a high-impact journal and later published in either a low- or high-impact journal differ?. Scientometrics 83 2 493–506 .
-
Chung, CK, Pennebaker, JW 2007 The psychological function of function words K Fiedler eds. Social communication Psychology Press New York 343–359.
-
Fast, LA, Funder, DC 2008 Personality as manifest in word use: Correlations with self-report, acquaintance report, and behavior. Journal of Personality and Social Psychology 94 2 334–346 .
-
Fraser, V. J., & Martin, J. G. (2009). Marketing data: Has the rise of impact factor led to the fall of objective language in the scientific article? Respiratory Research, 10. doi: .
-
Harnad, S. (2000). The invisible hand of peer review. Exploit Interactive (5). http://www.exploit-lib.org/issue5/peer-review/.
-
Hartley, J, Betts, L 2009 Common weaknesses in traditional abstracts in the social sciences. Journal of the American Society for Information Science and Technology 60 10 2010–2018 .
-
Hartley, J, Pennebaker, JW, Fox, C 2003 Abstracts, introductions and discussions: How far do they differ in style?. Scientometrics 57 3 389–398 .
-
Kahn, JH, Tobin, RM, Massey, AE, Anderson, JA 2007 Measuring emotional expression with the Linguistic Inquiry and Word Count. American Journal of Psychology 120 2 263–286.
-
Koonin, E., & Lipman, D. (2006). Systems: Reviving a culture of scientific debate. Can ‘open peer review’ work for biologists? Biology Direct is hopeful. Retrieved 21 June 2006, from http://www.nature.com/nature/peerreview/debate/nature05005.html.
-
Mungra, P, Webber, P 2010 Peer review process in medical research publications: Language and content comments. English for Specific Purposes 29 1 43–53 .
-
Oberlander, J, Gill, AJ 2006 Language with character: A stratified corpus comparison of individual differences in e-mail communication. Discourse Processes 42 3 239–270 .
-
Pennebaker, JW, Chung, CK, Ireland, M, Gonzales, A, Booth, RJ 2007 The development and psychometric properties of LIWC 2007 University of Texas/University of Auckland Austin, TX/Auckland.
-
Pennebaker, JW, King, LA 1999 Linguistic styles: Language use as an individual difference. Journal of Personality and Social Psychology 77 6 1296–1312 .
-
Pennebaker, JW, Mehl, MR, Niederhoffer, KG 2003 Psychological aspects of natural language use: Our words, our selves. Annual Review of Psychology 54:547–577 .
-
Popping, R 2000 Computer-assisted text analysis Sage London.
-
Pöschl, U 2004 Interactive journal concept for improved scientific publishing and quality assurance. Learned Publishing 17 2 105–113 .
-
Pöschl, U 2010 Interactive open access publishing and peer review: The effectiveness and perspectives of transparency and self-regulation in scientific communication and evaluation. Liber Quarterly 19 3/4 293–314.
-
StataCorp. 2011 Stata statistical software: Release 12 Stata Corporation College Station, TX.
-
Tausczik, YR, Pennebaker, JW 2010 The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology 29 1 24–54 .
Monthly Content Usage
| Abstract Views | Full Text Views | PDF Downloads | |
|---|---|---|---|
| Aug 2020 | 11 | 0 | 0 |
| Sep 2020 | 3 | 0 | 0 |
| Oct 2020 | 3 | 0 | 0 |
| Nov 2020 | 7 | 5 | 1 |
| Dec 2020 | 20 | 0 | 0 |
| Jan 2021 | 4 | 0 | 0 |
| Feb 2021 | 0 | 0 | 0 |
Copyright Akadémiai Kiadó
AKJournals is the trademark of Akadémiai Kiadó's journal publishing business branch.
Copyright Akadémiai Kiadó AKJournals is the trademark of Akadémiai Kiadó's journal publishing business branch.
Powered by PubFactory