In economic and social sciences it is crucial to test theoretical models against large and reliable databases. The general research challenge is to build up a well-structured database that suits the given research question well and that is cost efficient at the same time. In this paper we focus on crawler programs that proved to be an effective tool for data base building in very different problem settings. We present three structurally different research models where crawler programs can be applied successfully: exploration, classification and time series analysis. In the case of exploration we present findings about the Hungarian web agency industry where no previous statistical data was available about their operations. For classification we show how the top visited Hungarian web domains can be divided into predefined categories of e-business models. In the third research we used a crawler to gather the values of specific pre-defined records containing ticket prices of low cost airlines from one single site. Based on the experiences we highlight some conceptual conclusions and opportunities for crawler based research in e-business.
Bolisani, E. — Biagi Di, M. — Scarso, E. (2003): Knowledge Intermediation: New Business Models in the Digital Economy. Presented at the 16thInternational Bled e-Conference, Bled, Slovenia, 9–11 June.
Brin, S. — Page, L. (1998): The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 30(1–7): 107–117.
Page L., 'The anatomy of a large-scale hypertextual Web search engine' (1998) 30Computer Networks and ISDN Systems: 107-117.
Page L.The anatomy of a large-scale hypertextual Web search engineComputer Networks and ISDN Systems199830107117)| false
O’Boyle, P. — Acton, T. — Campion, M. — Conboy, K. — Scott, M. (2009): Towards a Toolset for Intranet Evaluation. Presented at the 22ndInternational Bled e-Conference, Bled, Slovenia, 14–17 June.
Osterwalder, A. — Pigneur, Y. (2002): An e-Business Model Ontology for modelling e-Business. Presented at the 15thBled Electronic Commerce Conference, e-Reality: Constructing the e-Economy, Bled, Slovenia, 1–11 June.
Polanski, P. (2006): Intellectual Property Law versus Customs and Values of the Internet Community. Presented at the 19thInternational Bled e-Conference, Bled, Slovenia, 5–7 June.
Shkapenyuk, V. — Suel, T. (2002): Design and implementation of a high performance distributed web crawler. In: Proceedings of the 18th International Conference on Data Engineering (ICDE), San Jose, CA: IEEE CS Press, pp. 357–368.
Suel T., '', in Proceedings of the 18th International Conference on Data Engineering (ICDE), (2002) -.
Suel T.Proceedings of the 18th International Conference on Data Engineering (ICDE)2002)| false