Authors:Primož Južnič, Stojan Pečlin, Matjaž Žaucer, Tilen Mandelj, Miro Pušnik, and Franci Demšar
The paper discusses the role of scientometric indicators in peer-review selection of research project proposals. An ex post
facto evaluation was made of three calls for research project proposals in Slovenia: 2003 with a peer review system designed
in a way that conflict of interest was not avoided effectively, 2005 with a sound international peer-review system with minimized
conflict of interest influence but a limited number of reviewers, and 2008 with a combination of scientometric indicators
and a sound international peer review with minimized conflict of interest influence. The hypothesis was that the three different
peer review systems would have different correlations with the same set of scientometric indicators. In the last two decision-making
systems (2005 and 2008) where conflict of interest was effectively avoided, we have a high percentage (65%) of projects that
would have been selected in the call irrespective of the method (peer review or bibliometrics solely). In contrast, in the
2003 call there is a significantly smaller percentage (49%) of projects that would have been selected in the call irrespective
of the method (peer review or bibliometrics solely). It was shown that while scientometric indicators can hardly replace the
peer-review system as the ultimate decision-making and support system, they can reveal its weaknesses on one hand and on the
other can verify peer-review scores and minimize conflict of interest if necessary.
Authors:Stojan Pečlin, Primož Južnič, Rok Blagus, Mojca Čižek Sajko, and Janez Stare
This study examines the effect of international collaboration of Slovenian authors and the status of journals where papers are published (as determined by their impact factors) on the impact of papers as measured by the number of citations papers receive. Research programme groups working in Slovenia in the 2004–2008 period in the fields of physics, chemistry, biology, biotechnology, and medical science were used for analyses. The results of the analyses show that the effects of the two factors differ among the fields. We discuss possible reasons for this, including the possibility that differences are the result of Slovenia's science policy.