View More View Less
  • 1 Department of Pediatrics, Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brazil
  • | 2 Department of Statistics, Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brazil
  • | 3 Programa de Pós-Graduação em Ciências da Saúde, Universidade Estadual de Montes Claros, Montes Claros, MG, Brazil
  • | 4 Rua Engenheiro Amaro Lanari, 389/501, 30310-580 Belo Horizonte, MG, Brazil
Restricted access

Abstract

Quantifying the relative performance of individual scholars has become an integral part of decision-making in research policy. The objective of the present study was to evaluate if the scholarship rank of Brazilian Council for Scientific and Technological Development (CNPq) researchers in Medicine is consistent with their scientific productivity. The Lattes curricula of 411 researchers (2006–2008) were included in the study. Scholarship category was the variable of interest. Other variables analyzed were: time since receiving the doctorate, teaching activity (undergraduate, master's and doctoral students), number of articles published, and number of papers indexed by the Institute for Scientific Information (ISI) and Scopus databases. Additional performance indicators included were: citations, h-index, and m-index. There was a significant difference among scholarship categories regarding number of papers per year, considering the entire scientific career (P < 0.001) or the last 5 years (P < 0.001). There was no significant difference among scholarship categories regarding the number of citations per article in the ISI (Thomson Reuters) database (P = 0.23). There was a significant difference in h-index among scholarship categories in both databases, i.e. (P < 0.001) and Scopus (P < 0.001). Regarding the m-index, there was a significant difference among categories only in the ISI database (P = 0.012). According to our findings, a better instrument for qualitative and quantitative indicators is needed to identify researchers with outstanding scientific output.

  • Abramo, G, D’Angelo, CA, Costa, F 2011 National research assessment exercises: The effects of changing the rules of the game during the game. Scientometrics 88:229238 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Abramo, G., D’Angelo, C. A., & Costa, F. (2011b). Research productivity: Are higher academic ranks more productive than lower ones? Scientometrics. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Allen, L, Jones, C, Dolby, K, Lynn, D, Walport, M 2009 Looking for landmarks: The role of expert review and bibliometric analysis in evaluating scientific publication outputs. PLoS One 4 6 e5910 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Arruda, D, Bezerra, F, Neris, VA, Toro, PR, Wainer, J 2009 Brazilian computer science research: Gender and regional distributions. Scientometrics 79 3 651665 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Barata, RB, Goldbaum, M 2003 A profile of researchers in public health with productivity grants from the Brazilian National Research Council (CNPq). Cad Saude Publica 19 6 18631876 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Boell, SK, Wilson, CS 2010 Journal impact factors for evaluating scientific performance: Use of h-like indicators. Scientometrics 82:613626 .

  • Bornmann, L, Daniel, H-D 2007 What do we know about the h index?. Journal of the American Society for Information Science 58:13811385 .

  • Bornmann, L, Mutz, R, Daniel, H-D 2008 Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science & Technology 59 5 830837 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Mutz, R, Neuhaus, C, Daniel, H-D 2008 Citation counts for research evaluation: Standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics 8:93102 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Wallon, G, Ledin, A 2008 Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data. Research Evaluation 17 2 149156 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Browman, HI, Stergiou, KI 2008 Factors and indices are one thing, deciding who is scholarly, why they are scholarly, and the relative value of their scholarship is something else entirely. Ethics in Science and Environmental Politics 8:13 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Burrell, QL 2007 Hirsch index or Hirsch rate? Some thoughts arising from Liang's data. Scientometrics 73 1 1928 .

  • Burrell, QL 2007 Hirsch's h-index: A stochastic model. Journal of Informetrics 1 1 1625 .

  • Cavalcante, RA, Barbosa, DR, Bonan, PRF, Pires, MBO, Martelli-Junior, H 2008 Perfil dos pesquisadores da área de odontologia no Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). Revista Brasileira de Epidemiologia 11 1 106113 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • CNPq, Ministério de Ciência e Tecnologia. (2010). Critérios de Julgamento—CA-MD. http://www.cnpq.br/cas/ca-md.htm#criterios. Accessed 4th Aug 2010.

    • Search Google Scholar
    • Export Citation
  • Franceschini, F, Maisano, D 2011 Proposals for evaluating the regularity of a scientist's research output. Scientometrics 88:279295 .

  • Geuna, A, Martin, BR 2003 University research evaluation and funding: An international comparison. Minerva 41:277304 .

  • Glanzel, W 2006 On the h-index: A mathematical approach to a new measure of publication activity and citation impact. Scientometrics 67:315321 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Glänzel, W, Schubert, A 2003 A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics 56:357367 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Haeffner-Cavaillon, N, Graillot-Gak, C 2009 The use of bibliometric indicators to help peer-review assessment. Archivum Immunologiae et Therapiae Experimentalis (Warsz) 57 1 3338 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hirsch, JE 2005 An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America 102 46 1656916572 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jeang, KT 2008 H-index, mentoring-index, highly-cited and highly-accessed: How to evaluate scientists?. Retrovirology 5:106 .

  • Lane, J 2010 Let's make science metrics more scientific. Nature 464 7288 488489 .

  • Lehmann, S, Jackson, AD, Lautrup, BE 2006 Measures for measures. Nature 444 7122 10031004 .

  • Leite, P, Mugnaini, R, Leta, J 2011 A new indicator for international visibility: Exploring Brazilian scientific community. Scientometrics 88:311319 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Martelli-Junior, H, Martelli, DR, Quirino, IG, Oliveira, MC, Lima, LS, Oliveira, EA 2010 CNPq researchers in medicine: A comparative study of research areas. Revista Da Associacao Medica Brasileira 56 4 478483 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mendes, P. H. C., Martelli, D. R., Souza, W. P., Filho, S. Q., & Martelli Junior, H. (2010). Perfil dos pesquisadores bolsistas de produtividade científica na medicina no CNPq, Brasil. Revista Brasileira de Educação Médica (in press).

    • Search Google Scholar
    • Export Citation
  • Moed, HF 2009 New developments in the use of citation analysis in research evaluation. Arch Immunol Ther Exp (Warsz) 57 1 1318 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Montgomery, D. C. (2008). Design and analysis of experiments (Vol. 1, 7th ed.). Boston: Wiley.

  • Oliveira, EA, Pecoits-Filho, R, Quirino, IG, Oliveira, MCL, Martelli, DR, Lima, LS et al. 2011 Perfil e produção científica dos pesquisadores do CNPq nas áreas de Nefrologia e Urologia. Jornal Brasileiro de Nefrologia 33:3137 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Oliveira, E. A., Ribeiro, A. L. P., Quirino, I. G., Oliveira, M. C., Martelli, D. R., Lima, L. S., et al. (2011b). Perfil e produção científica dos pesquisadores do Conselho Nacional de Desenvolvimento Científico e Tecnológico na área de Cardiologia. Arquivos Brasileiro de Cardiologia. doi: .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Panaretos, J, Malesios, C 2009 Assessing scientific research performance and impact with single indices. Scientometrics 81 3 635670 .

  • Petherick, A 2010 High hopes for Brazilian science. Nature 465 7299 674675 .

  • Podlubny, I 2005 Comparison of scientific impact expressed by the number of citations in different fields of science. Scientometrics 64:9599 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Regalado, A. (2010). Science in Brazil. Brazilian science: Riding a gusher. Science, 330 (6009), 13061312. doi: .

  • Santos, NCF, Candido, LFO, Kuppens, CL 2010 Produtividade em pesquisa do CNPq: Análise do perfil dos pesquisadores da química. Quimica Nova 33 2 489495 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Santos, SMC, Lima, LS, Martelli, DRB, Martelli Junior, H 2009 Perfil dos pesquisadores da Saúde Coletiva no Conselho Nacional de Desenvolvimento Científico e Tecnológico. Physis Revista de Saúde Coletiva 19 3 761775 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shibayama, S 2011 Distribution of academic research funds: A case of Japanese national research grant. Scientometrics 88:4360 .

  • AFJ van Raan 2006 Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics 67:491502.

    • Search Google Scholar
    • Export Citation

To see the editorial board, please visit the website of Springer Nature.

Manuscript submission: http://www.editorialmanager.com/scim/

For subscription options, please visit the website of Springer Nature.

Scientometrics
Language English
Size B5
Year of
Foundation
1978
Volumes
per Year
1
Issues
per Year
12
Founder Akadémiai Kiadó
Founder's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Publisher Akadémiai Kiadó
Springer Nature Switzerland AG
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
CH-6330 Cham, Switzerland Gewerbestrasse 11.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 0138-9130 (Print)
ISSN 1588-2861 (Online)