View More View Less
  • 1 Southern Cross University, PO Box 157, Lismore, NSW 2480, Australia
  • | 2 Max Planck Society, Administrative Headquarters, PO Box 101062, Munich 80084, Germany
Restricted access

Abstract

Excellence for Research in Australia (ERA) is an attempt by the Australian Research Council to rate Australian universities on a 5-point scale within 180 Fields of Research using metrics and peer evaluation by an evaluation committee. Some of the bibliometric data contributing to this ranking suffer statistical issues associated with skewed distributions. Other data are standardised year-by-year, placing undue emphasis on the most recent publications which may not yet have reliable citation patterns. The bibliometric data offered to the evaluation committees is extensive, but lacks effective syntheses such as the h-index and its variants. The indirect H2 index is objective, can be computed automatically and efficiently, is resistant to manipulation, and a good indicator of impact to assist the ERA evaluation committees and to similar evaluations internationally.

  • Abramo, G, D’Angelo, CA 2011 Evaluating research: from informed peer review to Bibliometrics. Scientometrics 87:499514 .

  • Abramo, G, D’Angelo, CA, Caprasecca, A 2009 Allocative efficiency in public research funding: Can bibliometrics help?. Research Policy 38 1 206215 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Alonso, S, Cabrerizo, FJ, Herrera-Viedma, E, Herrera, F 2009 h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics 3:273289 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anon. (2011). List of forestry universities and colleges. Wikipedia, http://en.wikipedia.org/wiki/List_of_forestry_universities_and_colleges [August 5, 2011].

    • Search Google Scholar
    • Export Citation
  • ANZSRC (2008). Australian and New Zealand Standard Research Classification. http://www.abs.gov.au/Ausstats/abs@.nsf/0/6BB427AB9696C225CA2574180004463E [August 5, 2011].

    • Search Google Scholar
    • Export Citation
  • ARC (2010a). The Excellence in Research for Australia (ERA) Initiative. http://www.arc.gov.au/era/default.htm [August 5, 2011].

  • ARC (2010b). Ranked Journal List. Australian Research Council. http://www.arc.gov.au/xls/ERA2010_journal_title_list.xls [August 5, 2011].

    • Search Google Scholar
    • Export Citation
  • ARC (2011a). What is the change to the journal indicator? Australian Research Council. http://www.arc.gov.au/era/faq.htm#2012 [August 5, 2011].

    • Search Google Scholar
    • Export Citation
  • ARC (2011b). ERA 2010 National Report. Australian Research Council, p. 320. http://www.arc.gov.au/pdf/ERA_report.pdf [August 5, 2011].

  • ARC (2011c). ERA 2010 citation benchmark methodology. Australian Research Council, p. 21. http://www.arc.gov.au/pdf/era_2010_citation_benchmark_methods.pdf [August 5, 2011].

    • Search Google Scholar
    • Export Citation
  • ARC (2011d). ERA 2010 results by field of research code. Australian Research Council. http://www.arc.gov.au/era/outcomes_2010/FoRindex [August 5, 2011].

    • Search Google Scholar
    • Export Citation
  • Arencibia-Jorge, R, Barrios-Almaguer, I, Fernández-Hernández, S, Carvajal-Espino, R 2008 Successive H indices and its applying in the institutional evaluation: a case study. Journal of the American Society for Information Science and Technology 59 1 155157 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Atkinson, R. & McLoughlin, C. (2011). Dawn of a new ERA? Australasian Journal of Educational Technology, 27 (3), iii-viii. http://ascilite.org.au/ajet/ajet27/editorial27-3.html.

    • Search Google Scholar
    • Export Citation
  • Bornmann, L 2010 Mimicry in science?. Scientometrics 86 1 173177 .

  • Bornmann, L 2011 Scientific peer review. Annual Review of Information Science and Technology 45:199245.

  • Bornmann, L F de Moya Anegón Leydesdorff, L 2010 Do scientific advancements lean on the shoulders of giants? A bibliometric investigation of the Ortega hypothesis. PLoS ONE 5 10 e13327 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Mutz, R, Daniel, H-D 2008 Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from Biomedicine. Journal of the American Society for Information Science and Technology 59 5 830837 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L, Mutz, R, Hug, SE, Daniel, H-D 2011 A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics 5:346359 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bornmann, L., Schier, H., Marx, W., & Daniel, H.-D. (2011b). Does the h index for assessing single publications really work? A case study on papers published in chemistry. Scientometrics, 89 (3), 835843.

    • Search Google Scholar
    • Export Citation
  • Carr, K. (2009). A message from the Minister. Australian Research Council, Discovery Newsletter, Autumn 2009. http://www.arc.gov.au/rtf/discovery_autumn09.rtf.

    • Search Google Scholar
    • Export Citation
  • Cooper, S, Poletti, A 2011 The new ERA of journal ranking. Australian Universities’ Review 53:5765.

  • Donovan, C, Butler, L 2007 Testing novel quantitative indicators of research ‘quality’, esteem and ‘user engagement’: An economics pilot study. Research Evaluation 16 4 231242 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Egghe, L 2008 Modelling successive h-indices. Scientometrics 77 3 377387 .

  • Egghe, L 2011 The single publication H-index and the indirect H-index of a researcher. Scientometrics 88 3 10031004 .

  • Falagas, ME, Pitsouni, EI, Malietzis, GA, Pappas, G 2008 Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. The FASEB Journal 22:338342 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Haddow, G, Genoni, P 2010 Citation analysis and peer ranking of Australian social science journals. Scientometrics 85:471487 .

  • Hicks, D 2009 Evolving regimes of multi-university research evaluation. Higher Education 57:393404 .

  • Hirsch, JE 2005 An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences 102:1656916572 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jones, D. (2011). Excellence in Research for Australia—a new ERA. Research Trends 23, 1011. http://www.researchtrends.com/wp-content/uploads/2011/06/Research_Trends_Issue23.pdf.

    • Search Google Scholar
    • Export Citation
  • Laband, DN, Zhang, D 2006 Citations, publications, and perceptions-based rankings of the research impact of North American forestry programs. Journal of Forestry 104 5 162254.

    • Search Google Scholar
    • Export Citation
  • Labbé, C 2010 Ike Antkare, one of the great stars in the scientific firmament. ISSI Newsletter 6 2 4852.

  • Lamp, J. (2011). 2010 finalised journals in an ANZ Field of Research: 0705 Forestry Sciences. http://lamp.infosys.deakin.edu.au/era/?page=jfordet10&selfor=0705 [5 August 2011].

    • Search Google Scholar
    • Export Citation
  • Leydesdorff, L, Bornmann, L, Mutz, R, Opthof, T 2011 Turning the tables in citation analysis one more time: principles for comparing sets of documents. Journal of the American Society of Information Science and Technology 62 7 13701381 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, J, Burnham, JF, Lemley, T, Britton, RM 2010 Citation Analysis: Comparison of Web of Science, Scopus, SciFinder, and Google Scholar. Journal of Electronic Resources in Medical Libraries 7 3 196217 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meho, LI, Yang, K 2007 Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar. Journal of the American Society for Information Science and Technology 58:21052125 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moed, HF 2009 New developments in the use of citation analysis in research evaluation. Archivum Immunologiae et therapiae Experimentalis 57:1318 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moosa, I 2011 The demise of the ARC journal ranking scheme: an ex post analysis of the accounting and finance journals. Accounting & Finance 51 3 809836 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moya-Anegón, FD, Chinchilla-Rodríguez, Z, Vargas-Quesada, B, Corera-Álvarez, E, Muñoz-Fernández, FJ, González-Molina, A, Herrero-Solana, V 2007 Coverage analysis of Scopus: A journal metric approach. Scientometrics 73:5378 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Norris, M, Oppenheim, C 2010 The h-index: a broad review of a new bibliometric indicator. Journal of Documentation 66 5 681705 .

  • Northcott, D, Linacre, S 2010 Producing spaces for academic discourse: The impact of research assessment exercises and journal quality rankings. Australian Accounting Review 20:3854 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pontille, D, Torny, D 2010 The controversial policies of journal ratings: Evaluating social sciences and humanities. Research Evaluation 15:347360 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Radicchi, F., & Castellano, C. (2011). Rescaling citations of publications in physics. Physical Review E, 83 (4), 6. doi: .

  • Rousseau, R, Yang, L, Yue, T 2010 A discussion of Prathap's h2-index for institutional evaluation with an application in the field of HIV infection and therapy. Journal of Informetrics 4 2 175184 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ruane, F, Tol, RSJ 2008 Rational (successive) h-indices: An application to economics in the Republic of Ireland. Scientometrics 75 2 395405 .

  • Runeson, G 2011 The demise of the journal ranking: a victory for common sense. Australasian Journal of Construction Economics and Building 11 2 99100.

    • Search Google Scholar
    • Export Citation
  • Schreiber, M 2010 Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers. Annalen der Physik 522 8 536554 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schubert, A 2007 Successive h-indices. Scientometrics 70 1 201205 .

  • Schubert, A 2009 Using the h-index for assessing single publications. Scientometrics 78 3 559565 .

  • Serenko, A, Dohan, M 2011 Comparing the expert survey and citation impact journal ranking methods: Example from the field of Artificial Intelligence. Journal of Informetrics 5 4 629648 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sinclair, R. (2010). RD&E strategy for the forest and wood products sector. Australia: Forest and Wood Products, p. 57. www.daff.gov.au/__data/assets/word_doc/0010/1770751/forest.doc.

    • Search Google Scholar
    • Export Citation
  • Singh, G, Haddad, KM, Chow, CW 2007 Are articles in “top” management journals necessarily of higher quality?. Journal of Management Inquiry 16 4 319331 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Slyder, JB, Stein, BR, Sams, BS, Walker, DM, Beale, BJ, Feldhaus, JJ, Copenheaver, CA 2011 Citation pattern and lifespan: A comparison of discipline, institution and individual. Scientometrics 89 3 955966 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Svantesson, DJB, White, P 2009 Entering an era of research ranking—will innovation and diversity survive?. Bond Law Review 21 3 7.

  • Taylor, J 2011 The assessment of research quality in UK universities: Peer review or metrics?. British Journal of Management 22:202217 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thor, A. & Bornmann, L. (2011). The calculation of the single publication h index and related performance measures: A web application based on Google Scholar data. Online Information Review, 35 (2), 291300. (Utility available here: http://labs.dbs.uni-leipzig.de/gsh/).

    • Search Google Scholar
    • Export Citation
  • Vanclay, JK 2008 Ranking forestry journals using the h-index. Journal of Informetrics 2:326334 .

  • Vanclay, JK 2011 An evaluation of the Australian Research Council's journal ranking. Journal of Informetrics 5:265274 .

  • Vanclay, JK 2012 What was wrong with Australia's journal ranking?. Journal of Informetrics 6:5354 .

Manuscript submission: http://www.editorialmanager.com/scim/

  • Impact Factor (2019): 2.867
  • Scimago Journal Rank (2019): 1.210
  • SJR Hirsch-Index (2019): 106
  • SJR Quartile Score (2019): Q1 Computer Science Apllications
  • SJR Quartile Score (2019): Q1 Library and Information Sciences
  • SJR Quartile Score (2019): Q1 Social Sciences (miscellaneous)
  • Impact Factor (2018): 2.770
  • Scimago Journal Rank (2018): 1.113
  • SJR Hirsch-Index (2018): 95
  • SJR Quartile Score (2018): Q1 Library and Information Sciences
  • SJR Quartile Score (2018): Q1 Social Sciences (miscellaneous)

For subscription options, please visit the website of Springer

Scientometrics
Language English
Size B5
Year of
Foundation
1978
Volumes
per Year
4
Issues
per Year
12
Founder Akadémiai Kiadó
Founder's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Publisher Akadémiai Kiadó
Springer Nature Switzerland AG
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
CH-6330 Cham, Switzerland Gewerbestrasse 11.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 0138-9130 (Print)
ISSN 1588-2861 (Online)