In this study the issue of the validity of the argument against the applied length of citation windows in Journal Impact Factors calculations is critically re-analyzed. While previous studies argued against the relatively short citation window of 1–2 years, this study shows that the relative short term citation impact measured in the window underlying the Journal Impact Factor is a good predictor of the citation impact of the journals in the next years to come. Possible exceptions to this observation relate to journals with relatively low numbers of publications, and the citation impact related to publications in the year of publication. The study focuses on five Journal Subject Categories from the science and social sciences, on normal articles published in these journals, in the 2 years 2000 and 2004.
The paper discusses an application of bibliometric techniques in the social sciences. While the interest of policy makers
is growing, the topic is getting more and more attention from bibliometricians. However, many efforts are put into developing
tools to measure scientific output and impact outside the world of the Social Sciences Citation Index, while the use of the
SSCI for bibliometric applications is covered with obscurity and myths. This study attempts to clarify some of the topics
mentioned against the application of the SSCI for evaluation purposes. The study will cover topics like the existing publication
and citation culture within the social sciences, the effect of variable citation windows, and the (geographical) origin of
In this study we have focused on long term developments of various types of scientific publishing, and the field-normalized
impact generated by these various types. The types of scientific output distinguished are output resulting from international
cooperation, national cooperation, and single address publications, in which no apparent cooperation is found. A fourth type
is distinguished by focusing on first authorship, within the international cooperation output. Changes in especially the share
of a country’s output from first-authored international cooperation and the share of single address publications can be regarded
as indicators of strength and/or weakness of a science system.
This paper discusses development and application of journal impact indicators in a number of bibliometric studies commissioned
by Dutch organizations and institutions, and conducted in our institute during the past five years. An outline is given of
the research questions addressed in these studies and their policy context. For each study the appropriateness of the use
of journal impact indicators produced by the Institute for Scientific Information (ISI) is evaluated. Alternative journal
impact measures were developed which are shown to be more appropriate in the particular research and policy contexts than
the ISI measures. These measures were considered to be highly useful by the users. The studies have revealed methodological
flaws of the ISI journal impact factors.
Authors:Rodrigo Costas, Thed van Leeuwen and María Bordons
This paper focuses on the study of self-citations at the meso and micro (individual) levels, on the basis of an analysis of
the production (1994–2004) of individual researchers working at the Spanish CSIC in the areas of Biology and Biomedicine and
Material Sciences. Two different types of self-citations are described: author self-citations (citations received from the
author him/herself) and co-author self-citations (citations received from the researchers’ co-authors but without his/her
participation). Self-citations do not play a decisive role in the high citation scores of documents either at the individual
or at the meso level, which are mainly due to external citations. At micro-level, the percentage of self-citations does not
change by professional rank or age, but differences in the relative weight of author and co-author self-citations have been
found. The percentage of co-author self-citations tends to decrease with age and professional rank while the percentage of
author self-citations shows the opposite trend. Suppressing author self-citations from citation counts to prevent overblown
self-citation practices may result in a higher reduction of citation numbers of old scientists and, particularly, of those
in the highest categories. Author and co-author self-citations provide valuable information on the scientific communication
process, but external citations are the most relevant for evaluative purposes. As a final recommendation, studies considering
self-citations at the individual level should make clear whether author or total self-citations are used as these can affect
Authors:Clara Calero, Thed van Leeuwen and Robert Tijssen
Bio-pharmaceutical R&D is increasingly an international affair. Research articles published in the peer-reviewed international
scientific and technical journals represent quantifiable research outputs of bio-pharmaceutical firms. Large-scale systemic
measurements of worldwide trends and sectoral patterns within bio-pharmaceutical science can be gauged from these articles,
where coauthored research papers are assumed to reflect research cooperation and associated knowledge flows and exchanges.
We focus our attention on the largest science-based multinational enterprises (MNEs), those that produce relatively large
quantities of research articles. The study deals with the worldwide output of research articles that are co-produced by corporate
researchers during the years 1996–2001.
We employ these publications to examine structural factors characterizing research cooperation networks within industry at
the level of major geographical regions (North America, Europe, Pacific-Asia), with a breakdown by within-MNE and between-MNE
network linkages. The descriptive statistics on publication output and results of network analyses of co-publication linkages
not only indicate regional differences, with a central role for US companies in biopharmaceutical research, but also a variety
of firm-specific research cooperation networks which enabled us to develop a tentative typology of MNEs in terms of their
intra- and interorganizational patterns of research cooperation linkages.
Authors:Robert Tijssen, Martijn Visser and Thed van Leeuwen
This paper introduces a citation-based "systems approach" for analyzing the various institutional and cognitive dimensions
of scientific excellence within national research systems. The methodology, covering several aggregate levels, focuses on
the most highly cited research papers in the international journal literature. The distribution of these papers across institutions
and disciplines enables objective comparisons their (possible) international-level scientific excellence. By way of example,
we present key results from a recent series of analyses of the research system in the Netherlands in the mid 1990s, focussing
on the performance of the universities across the various major scientific disciplines within the context of the entire system"s
scientific performance. Special attention is paid to the contribution in the world"s top 1% and top 10% most highly cited
research papers. The findings indicate that these high performance papers provide a useful analytical framework - both in
terms of transparency, cognitive and institutional differentiation, as well as its scope for domestic and international comparisons
- providing new indicators for identifying "world class" scientific excellence at the aggregate level. The average citation
scores of these academic "Centres of Scientific Excellence" appear to be an inadequate predictor of their production of highly
cited papers. However, further critical reflection and in-depth validation studies are needed to establish the true potential
of this approach for science policy analyses and evaluation of research performance.
Summary In this study, journal impact factors play a central role. In addition to this important bibliometric indicator, which evolves around the average impact of a journal in a two-year timeframe, related aspects of journal impact measurement are studied. Aspects like the output volume, the percentage of publications not cited, and the citation frequency distribution within a set timeframe are researched, and put in perspective with the 'classical' journal Impact Factor. In this study it is shown that these aspects of journal impact measurement play a significant role, and are strongly inter-related. Especially the separation between journals on the basis of the differences in output volume seems to be relevant, as can be concluded from the different results in the analysis of journal impact factors, the degree of uncitedness, and the share of a journal its contents above or below the impact factor value.
Authors:Anton Nederhof, Thed van Leeuwen and Anthony van Raan
In this study we show that it is possible to identify top-cited publications other than Web of Science (WoS) publications,
particularly non-journal publications, within fields in the social and behavioral sciences. We analyzed references in publications
that were themselves highly cited, with at least one European address. Books represent between 62 (psychology) and 81% (political
science) of the non-WoS references, journal articles 15–24%. Books (economics, political science) and manuals (psychology)
account for the most highly cited publications. Between 50 (psychology, political science) and 71% (economics) of the top-ranked
most cited publications originated from the US versus between 18 (economics) and 38% (psychology) from Europe. Finally, it
is discussed how the methods and procedures of the study can be optimized.
Authors:Ed Rinia, Thed van Leeuwen and Anthony van Raan
In an evaluation of physics research programs in the Netherlands, held in 1996, assessments of research by expert panels were
supplemented with bibliometric analysis. This latter analysis included the calculation of several bibliometric indicators,
among which some taking journal impact measures as a baseline. Final outcomes of this evaluation provided an opportunity to
re-examine the results of this assessment from the perspective of the degree of interdisciplinarity of programs involved.
In this paper we discuss results of this latter analysis, in particular with respect to the relation between several citation
based indicators and interdisciplinary research in Dutch physics.