This paper first explains the need to define subfields of science by means of “filters” that selectively retrieve papers from
a database, and then describes how such filters are constructed and calibrated. Good filters should have precision and recall
of the order of 90% so as to be representative of a subfield; they are created by an interactive partnership between an expert
in the subject and a bibliometrician. They are based primarily on the use of title keywords, often in combination rather than
singly, and specialist journals. Their calibration depends on experts marking lists of papers extracted by the filter as relevant,
don't know or not relevant. This allows the actual size of a subfield to be estimated and hence the relative importance accorded
to it within a major field of science. It permits organisations and countries to see their contributions to individual scientific
subfields in detail.
Authors:Miklós Arató, László Martinek, and Miklós Mályusz
, T. , Balabdaoui , F. and Raftery , A. E. , Probabilistic forecasts, calibration and sharpness , Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 69 ( 2 ) ( 2007 ) 243 – 268
; Waltman et al. 2011 ; Prathap 2011a ; Leydesdorff and Opthof 2011 ) indicate that there is confusion remaining on the key but separate steps required in evaluative bibliometrics. One is on the calibration for quality and whether there is a need for
Authors:Thomas Gurney, Edwin Horlings, and Peter van den Besselaar
ID pairing. The data was split into calibration and testing sets in an approximately 25:75 ratio to test the validity of the model. A regression was run with the NC codes as filters.
The full regression formula is as shown in Eq. 1 . For
papers. 3 Keeping the parameter values found during calibration, we experiment with varying the reference-selection method. The fitness of the best solution found during each simulation run is recorded. Figure 13 shows mean results for 200 simulation