Biblioteca Digital do Instituto Politécnico de Bragança   Instituto Politécnico de Bragança

Biblioteca Digital do IPB >
Escola Superior de Tecnologia e Gestão >
Informática e Comunicações >
IC - Artigos em Proceedings Não Indexados ao ISI >

Please use this identifier to cite or link to this item: http://hdl.handle.net/10198/526

Título: Geographical partition for distributed web crawling
Autor: Exposto, José
Macedo, Joaquim
Pina, António
Alves, Albano
Rufino, José
Palavras-chave: Web mining
Parallel crawling
Web partitioning
Issue Date: 2005
Editora: ACM
Citação: Exposto, José; Macedo, Joaquim; Herzog, Pina, António, Alves, Albano; Rufino, José (2005) - Geographical partition for distributed web crawling. In International Conference on Information and Knowledge Management. Bremen, Germany. ISBN 1-59593-140-6
Resumo: This paper evaluates scalable distributed crawling by means of the geographical partition of the Web. The approach is based on the existence of multiple distributed crawlers each one responsible for the pages belonging to one or more previously identified geographical zones. The work considers a distributed crawler where the assignment of pages to visit is based on page content geographical scope. For the initial assignment of a page to a partition we use a simple heuristic that marks a page within the same scope of the hosting web server geographical location. During download, if the analyze of a page contents recommends a different geographical scope, the page is forwarded to the well-located web server. A sample of the Portuguese Web pages, extracted during the year 2005, was used to evaluate: a) page download communication times and the b) overhead of pages exchange among servers. Evaluation results permit to compare our approach to conventional hash partitioning strategies.
Arbitragem científica: yes
URI: http://hdl.handle.net/10198/526
Versão do Editor: http://www.tzi.de/sites/CIKM2005/programs.html
Appears in Collections:IC - Artigos em Proceedings Não Indexados ao ISI

Files in This Item:

File Description SizeFormat
GIR2005-exp.pdf180,21 kBAdobe PDFView/Open
Statistics
FacebookTwitterDeliciousLinkedInDiggGoogle BookmarksMySpaceOrkut
Formato BibTex mendeley Endnote Logotipo do DeGóis 

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

 


  © Instituto Politécnico de Bragança - Biblioteca Digital - Feedback - Statistics
  Estamos no RCAAP Governo Português separator Ministério da Educação e Ciência   Fundação para a Ciência e a Tecnologia

Financiado por:

POS_C UE