View More View Less
  • 1 Faculty of Business, Laval University, Quebec City, QC G1V 0A6, Canada; rejean.landry@mng.ulaval.ca
Restricted access

Abstract

Research assessment carries important implications both at the individual and institutional levels. This paper examines the research outputs of scholars in business schools and shows how their performance assessment is significantly affected when using data extracted either from the Thomson ISI Web of Science (WoS) or from Google Scholar (GS). The statistical analyses of this paper are based on a large survey data of scholars of Canadian business schools, used jointly with data extracted from the WoS and GS databases. Firstly, the findings of this study reveal that the average performance of B scholars regarding the number of contributions, citations, and the h-index is much higher when performances are assessed using GS rather than WoS. Moreover, the results also show that the scholars who exhibit the highest performances when assessed in reference to articles published in ISI-listed journals also exhibit the highest performances in Google Scholar. Secondly, the absence of association between the strength of ties forged with companies, as well as between the customization of the knowledge transferred to companies and research performances of B scholars such as measured by indicators extracted from WoS and GS, provides some evidence suggesting that mode 1 and 2 knowledge productions might be compatible. Thirdly, the results also indicate that senior B scholars did not differ in a statistically significant manner from their junior colleagues with regard to the proportion of contributions compiled in WoS and GS. However, the results show that assistant professors have a higher proportion of citations in WoS than associate and full professors have. Fourthly, the results of this study suggest that B scholars in accounting tend to publish a smaller proportion of their work in GS than their colleagues in information management, finance and economics. Fifthly, the results of this study show that there is no significant difference between the contributions record of scholars located in English language and French language B schools when their performances are assessed with Google Scholar. However, scholars in English language B schools exhibit higher citation performances and higher h-indices both in WoS and GS. Overall, B scholars might not be confronted by having to choose between two incompatible knowledge production modes, but with the requirement of the evidence-based management approach. As a consequence, the various assessment exercises undertaken by university administrators, government agencies and associations of business schools should complement the data provided in WoS with those provided in GS.

  • Adler, NJ, Harzing, A-W. When knowledge wins: Transcending the sense and nonsense of academic rankings. Academy of Management Learning & Education 2009 8 1 7295 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Aram, JD PF Salipante Jr 2003 Bridging scholarship in management: Epistemological reflections. British Journal of Management 14:189205 .

  • Baneyz, A. “Publish or Perish” as citation metrics used to analyse scientific output in the humanities: Internal case studies in economics, geography, social science, philosophy, and history. Archivum Immunologiae et Therapiae Experimentalis 2008 56:363371 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bar-Ilan, J. Which h-index? A comparison of Wos, Scopus and Google Scholar. Scientometrics 2008 74 2 257271 .

  • Bartneck, C, Kokkelmans, S. Detecting h-index manipulation through self-citation analysis. Scientometrics 2011 87:8598 .

  • Bartunek, JM. What has happened to Mode 2?. British Journal of Management 2011 22:555558.

  • Bennis, W. G., & O'Toole, J. H. (2005). How Business Schools lost their way. Harvard Business, Review, May.

  • Bercovitz, J., & Feldman, M. (2003). Technology transfer and the academic department: who participates and why? Distinguished Lecture, DRUID Summer Conference, Copenhagen, 12–14 June 2002.

    • Search Google Scholar
    • Export Citation
  • Bozeman, B, Gaughan, M. Impacts of grants and contracts on academic researchers’ interactions with industry. Research Policy 2007 36:694707 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Burt, RS 1992 Structural holes. The social structure of competition Harvard University Press Cambridge.

  • Burt, RS 2005 Brokerage and closure: An introduction to social capital Oxford University Press Oxford.

  • Butler, L. (2006). RQF pilot study project—History and political science methodology for citation analysis. Accessed from http://www.chass.org.au/papers/pdf/PAP20061102LB.pdf.

    • Search Google Scholar
    • Export Citation
  • Carayol, N, Matt, M. Does research organization influence academic production? Laboratory level evidence from a large European university. Research Policy 2004 33 8 10811102 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Caswill, C, Wensley, R. Doors and boundaries: A recent history of the relationship between research and practice in UK organizational and management research. Business History 2007 49:293320 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • D'Este, P, Patel, P. University-industry linkages in the UK: What are the factors determining the variety of interactions with industry?. Research Policy 2007 36 9 12951313 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • D'Este, P, Perkmann, M. Why do academics engage with industry? The entrepreneurial university and individual motivations. Journal of Technology Transfer 2011 36 3 316339 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dillman, DA 2000 Mail and Internet surveys: The tailored design method John Wiley New York.

  • Dillman, DA, Bowker, DK 2001 The Web questionnaire challenge to survey methodologists UD Reips M Bosnjak eds. Dimensions of Internet science Pabst Science Publishers Lengerich 159178.

    • Search Google Scholar
    • Export Citation
  • Ennen, E, Richter, A. The whole is more than the sum of its parts—or is it? A review of the empirical literature on complementarities in organizations. Journal of Management 2010 36 1 207233 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gaddis, SE. How to design online surveys. Training and Development 1998 52:6772.

  • Gibbons, M, Limoges, C, Nowotny, H, Schwartzman, S, Scott, P, Trow, M 1994 The new production of knowledge: The dynamics of science and research in contemporary societies Sage London.

    • Search Google Scholar
    • Export Citation
  • Hansen, MT. The search-transfer problem: The role of weak ties in sharing knowledge across organization subunits. Administrative Science Quarterly 1999 44:82111 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harzing, A. W. (2007). Publish or Perish. available from: http://www.harzing.com/pop.htm.

  • Harzing, A-W 2010 The Publish or Perish Book: Your guide to effective responsible citation analysis Tarma Software Research Melbourne.

  • Harzing, AW R Van der Wal 2009 A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business?. Journal of American Society for Information Science and Technology 60 1 4146 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hirsch, J. An index to quantify an individual's scientific research output. Proceedings of the National academy of Sciences of the United States of America 2005 102 46 1656916572 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hughes, T, Bence, D, Grisoni, L, O'Regan, N, Wornham, D. Scholarship that matters: Academic-practitioner engagement in business and management. Academy of Management Learning & Education 2011 10 1 4057 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jacso, P. As we may search—Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Science 2005 89 9 15371547.

    • Search Google Scholar
    • Export Citation
  • Landry, R, Amara, N, Ouimet, M. Determinants of knowledge transfer: Evidence from Canadian University researchers in natural sciences and engineering. Journal of Technology Transfer 2007 32 6 561592 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Landry, R, Saïhi, M, Amara, N, Ouimet, M. Evidence on how academics manage their portfolio of knowledge transfer activities. Research Policy 2010 39:13861403 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meho, LI, Yang, K. Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology 2007 58 13 21052125 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Milgrom, P, Roberts, J. Complementarities and fit strategy, structure, and organiational change in manufacturing. Journal of Accounting & Economics 1995 19:179208 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mingers, J. Measuring the research contribution of management academics using the Hirsch-index. Journal of Operational Research Society 2008 60 8 11431153.

    • Search Google Scholar
    • Export Citation
  • Mingers, J., & Harzing, A. (2007). Ranking journals in business and management : A statistical analysis of the Harzing dataset. European Journal of Information Systems, 16 (4), 303316.

    • Search Google Scholar
    • Export Citation
  • Mingers, J, Lipitakis, EAECG. Counting the citations: a comparison of Web of Science and Google Scholar in the field of business and management. Scientometrics 2010 85:613625 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nicolau, N, Birley, S. Academic networks in a trichotomous categorisation of university spinouts. Journal of Business Venturing 2003 18 3 333359 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pettigrew, A. M. (1997). The double hurdles for management research. In T. Clarke (ed.), Advancement in organizational behavior: Essays in honour of derek S. Puh (pp. 277296). London: Darmouth Press.

    • Search Google Scholar
    • Export Citation
  • Pettigrew, AM. Scholarship with impact. Brisish Journal of Management. 2011 22:347354.

  • Ranga, LM, Debackere, K, Tunzelmann, NV. Entrepreneurial universities and the dynamics of academic knowledge production: A case study of basic vs. applied research in Belgium. Scientometrics 2003 58 2 301320 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reed, M. The theory/practice gap: A problem for research in business schools?. Journal of Management Development 2009 28 8 685693 .

  • Schmoch, U., Schubert, T., Jansen, D., Heidler, R., & von Görtz, R. (2010). How to use indicators to measure scientific performance: A balanced approach. Research Evaluation, 19 (1), 218.

    • Search Google Scholar
    • Export Citation
  • Starkey, K., & Madan, P. (2001). Bridging the relevance gap: Aligning stakeholders in the future of management research. British Journal of Management, 12, Special Issue-S3-S-26.

    • Search Google Scholar
    • Export Citation
  • Stephan, PE. The economics of science. Journal of Economic Literature 1996 34 3 11991235.

  • Teece, D. Business models, business strategy and innovation. Long Range Planning 2009 .

  • Thomas, H, Wilson, A. An analysis of the environment and competitive dynamics of management research. Journal of Management development 2009 28 8 668684 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Thorpe, R, Eden, C, Bessant, J, Ellwood, P. Rigour, relevance and reward: introducing the knowledge translation value-chain. British Journal of Management 2011 22:420431.

    • Search Google Scholar
    • Export Citation
  • Uzzi, B. Social structure and competition in interfirm networks: The paradox of embeddedness. Administrative Science Quarterly 1997 41:3367.

    • Search Google Scholar
    • Export Citation
  • WH Van de Ven Johnson, PE. Knowledge for theory and practice. Academy of Management Review 2006 31 4 802821 .

  • Van Looy, B., Callaert, J., & Debackere, K. (2006). Publication and patent behaviour of academic researchers: Conflicting, reinforcing or merely co-existing? Research Policy, 35(4), 596608.

    • Search Google Scholar
    • Export Citation