View More View Less
  • 1 Peter F. Bronfman Library, York University, 4700 Keele Street, Toronto, ON M3J 1P3, Canada
  • | 2 Statistical Cybermetrics Research Group, School of Technology, University of Wolverhampton, Wulfruna Street, Wolverhampton WV1 1LY, UK m.thelwall@wlv.ac.uk
  • | 3 Biomedical Branch Library, Gordon and Leslie Diamond Health Care Centre, University of British Columbia, 2775 Laurel Street, Floor 2, Vancouver BC V5Z 1M9, Canada dean.giustini@ubc.ca
Restricted access

Abstract

This paper investigates whether CiteULike and Mendeley are useful for measuring scholarly influence, using a sample of 1,613 papers published in Nature and Science in 2007. Traditional citation counts from the Web of Science (WoS) were used as benchmarks to compare with the number of users who bookmarked the articles in one of the two free online reference manager sites. Statistically significant correlations were found between the user counts and the corresponding WoS citation counts, suggesting that this type of influence is related in some way to traditional citation-based scholarly impact but the number of users of these systems seems to be still too small for them to challenge traditional citation indexes.

  • Aguillo, I. (2011). Is Google Scholar useful for bibliometrics? A webometrics analysis. Proceedings of the ISSI 2011 Conference. Presented at the 13th International Conference of the International Society for Scientometrics & Informetrics (pp. 1318), Durban, 4-7 July 2011.

    • Search Google Scholar
    • Export Citation
  • Belew, R. K. (2005). Scientific impact quantity and quality: Analysis of two sources of bibliographic data. Arxiv preprint cs/0504036.

  • Brody, T, Harnad, S, Carr, L 2006 Earlier web usage statistics as predictors of later citation impact. Journal of the American Society for Information Science and Technology 57 8 10601072 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Burgelman, J., Osimo, D., & Bogdanowicz, M. (2010). Science 2.0 (change will happen…). First Monday, 15 (7). Retrieved from http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2961/2573.

    • Search Google Scholar
    • Export Citation
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Mahwah: Lawrence Erlbaum.

  • Cronin, B 2001 Bibliometrics and beyond: Some thoughts on web-based citation analysis. Journal of Information Science 27 1 17 .

  • Cronin, B, Snyder, HW, Rosenbaum, H, Martinson, A, Callahan, E 1998 Invoked on the web. Journal of the American Society for Information Science 49 14 13191328 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ginsparg, P. (2007). Next-generation implications of Open Access. CTWatch Quarterly, 2 (3). Retrieved from http://www.ctwatch.org/quarterly/print.php?p=80.

    • Search Google Scholar
    • Export Citation
  • Groth, P., & Gurney, T. (2010). Studying scientific discourse on the Web using bibliometrics: A chemistry blogging case study—web science repository. Proceedings of the WebSci’10. Raleigh, NC. Retrieved April 26-27, 2010, from http://journal.webscience.org/308/.

    • Search Google Scholar
    • Export Citation
  • Henning, V. (2010). The top 10 journal articles published in 2009 by readership on Mendeley | Mendeley Blog. Retrieved August 8, 2010, from http://www.mendeley.com/blog/academic-features/the-top-10-journal-articles-published-in-2009-by-readership-on-mendeley/.

    • Search Google Scholar
    • Export Citation
  • Henning, V., & Reichelt, J. (2008). Mendeley—A Last.fm for research? In IEEE Fourth International Conference on eScience, 2008 (pp. 327328).

    • Search Google Scholar
    • Export Citation
  • Jiang, J., He, D., & Ni, C. (2011). Social reference: Aggregating online usage of scientific literature in CiteULike for clustering academic resources. Proceeding of the 11th annual international ACM/IEEE joint conference on Digital libraries (pp. 401402), Ottawa, Ontario, 13-17 June 2011.

    • Search Google Scholar
    • Export Citation
  • Kousha, K, Thelwall, M 2007 Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis. Journal of the American Society for Information Science and Technology 58 7 10551065 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kousha, K, Thelwall, M 2008 Assessing the impact of disciplinary research on teaching: An automatic analysis of online syllabuses. Journal of the American Society for Information Science and Technology 59 13 20602069 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kousha, K, Thelwall, M 2009 Google book search: Citation analysis for social science and the humanities. Journal of the American Society for Information Science and Technology 60 8 15371549 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kousha, K, Thelwall, M, Rezaie, S 2010 Using the Web for research evaluation: The Integrated Online Impact indicator. Journal of Informetrics 4 1 124135 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Li, X., Thelwall, M., & Giustini, D. (2011). Validating online reference managers for scholarly impact measurement. In E. Noyons, P. Ngulube & J. Leta (eds.), Proceedings of ISSI 2011—The 13th International Conference on Scientometrics and Informetrics (pp. 454462), Durban, 4-7 July 2011.

    • Search Google Scholar
    • Export Citation
  • Meho, LI, Yang, K 2007 Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology 58 13 21052125 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mendeley . (2010). Academic reference management software for researchers | Mendeley. Retrieved August 8, 2010, from http://www.mendeley.com/.

    • Search Google Scholar
    • Export Citation
  • Moed, HF 2005 Citation analysis in research evaluation Springer New York.

  • Neylon, C, Wu, S 2009 Article-level metrics and the evolution of scientific impact. PLoS Biol 7 11 e1000242 .

  • Norris, M, Oppenheim, C 2010 Peer review and the h-index: Two studies. Journal of Informetrics 4 3 221232 .

  • PLoS ONE . (2009a). Article-level metrics. Retrieved July 20, 2010, from http://article-level-metrics.plos.org/.

  • PLoS ONE . (2009b). New addition to article-level metrics—blog posts from ResearchBlogging.org | Public Library of Science. Retrieved July 20, 2010, from http://www.plos.org/cms/node/500.

    • Search Google Scholar
    • Export Citation
  • Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Presented at the American Society for Information Science & Technology Annual Meeting, Pittsburgh, PA, 22-27 October 2010.

    • Search Google Scholar
    • Export Citation
  • Priem, J., & Hemminger, B. M. (2010). Scientometrics 2.0: Toward new metrics of scholarly impact on the social Web. First Monday, 15 (7). Retrieved from http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2874/2570.

    • Search Google Scholar
    • Export Citation
  • Priem, J., Taraborelli, D., Groth, P. & Neylon, C. (2010). Alt-metrics: A manifesto. Retrieved July 25, 2011, from http://altmetrics.org/manifesto/.

    • Search Google Scholar
    • Export Citation
  • Research Information Network. (2010). If you build it, will they come? How researchers perceive and use web 2.0 | Research Information Network. Retrieved August 6, 2010, from http://www.rin.ac.uk/our-work/communicating-and-disseminating-research/use-and-relevance-web-20-researchers.

    • Search Google Scholar
    • Export Citation
  • Shepherd, P. T. (2007). Final report on the investigation into the feasibility of developing and implementing journal usage factors. Retrieved January 2, 2011, from http://www.uksg.org/sites/uksg.org/files/FinalReportUsageFactorProject.pdf.

    • Search Google Scholar
    • Export Citation
  • Smith, A. G. (2011). Wikipedia and institutional repositories: An academic symbiosis? Proceedings of the ISSI 2011 Conference. Presented at the 13th International Conference of the International Society for Scientometrics & Informetrics (pp. 794800), Durban, 4-7 July 2011.

    • Search Google Scholar
    • Export Citation
  • Taraborelli, D. (2008). Soft peer review: Social software and distributed scientific evaluation. In 8th International Conference on the Design of Cooperative Systems. Carry-le-Rouet: Institut d’Etudes Politiques d’Aix-en-Provence, Aix-en-Provence (pp. 99110), France.

    • Search Google Scholar
    • Export Citation
  • Taraborelli, D. (2010). ReaderMeter: Research impact, crowdsourced. Retrieved July 28, 2011, from http://readermeter.org/.

  • Thelwall, M 2008 Bibliometrics to webometrics. Journal of information science 34 4 605621 .

  • Thelwall, M, Kousha, K 2008 Online presentations as a source of scientific impact? An analysis of PowerPoint files citing academic journals. Journal of the American Society for Information Science and Technology 59 5 805815 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, L, Shaw, D 2003 Bibliographic and web citations: What is the difference?. Journal of the American Society for Information Science and Technology 54 14 13131322 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, L, Shaw, D 2005 Web citation data for impact assessment: a comparison of four science disciplines. Journal of the American Society for Information Science and Technology 56 10 10751087 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vaughan, L, Shaw, D 2008 A new look at evidence of scholarly citation in citation indexes and from web sources. Scientometrics 74 2 317330 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ware, M., & Monkman, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community-an international study. Publishing Research Consortium. Retrieved from .

    • Search Google Scholar
    • Export Citation
  • Weller, K., & Puschmann, C. (2011). Twitter for scientific communication: How can citations/references be identified and measured? Proceedings of the ACM WebSci’11 (pp. 14). Koblenz, Germany, 14-17 June 2011.

    • Search Google Scholar
    • Export Citation
  • Zaugg, H, West, RE, Tateishi, I, Randall, DL 2011 Mendeley: Creating communities of scholarly inquiry through research collaboration. TechTrends 55 1 3236 .

    • Crossref
    • Search Google Scholar
    • Export Citation

Manuscript submission: http://www.editorialmanager.com/scim/

  • Impact Factor (2019): 2.867
  • Scimago Journal Rank (2019): 1.210
  • SJR Hirsch-Index (2019): 106
  • SJR Quartile Score (2019): Q1 Computer Science Apllications
  • SJR Quartile Score (2019): Q1 Library and Information Sciences
  • SJR Quartile Score (2019): Q1 Social Sciences (miscellaneous)
  • Impact Factor (2018): 2.770
  • Scimago Journal Rank (2018): 1.113
  • SJR Hirsch-Index (2018): 95
  • SJR Quartile Score (2018): Q1 Library and Information Sciences
  • SJR Quartile Score (2018): Q1 Social Sciences (miscellaneous)

For subscription options, please visit the website of Springer

Scientometrics
Language English
Size B5
Year of
Foundation
1978
Volumes
per Year
4
Issues
per Year
12
Founder Akadémiai Kiadó
Founder's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Publisher Akadémiai Kiadó
Springer Nature Switzerland AG
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
CH-6330 Cham, Switzerland Gewerbestrasse 11.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 0138-9130 (Print)
ISSN 1588-2861 (Online)

Monthly Content Usage

Abstract Views Full Text Views PDF Downloads
Feb 2021 121 2 0
Mar 2021 95 2 0
Apr 2021 74 0 0
May 2021 93 0 0
Jun 2021 115 0 0
Jul 2021 141 0 0
Aug 2021 12 0 0