Untitled

Funder

Organizational Unit

Publications

Geographical partition for distributed web crawling

Publication . Exposto, José; Macedo, Joaquim; Pina, António; Alves, Albano; Rufino, José

This paper evaluates scalable distributed crawling by means of the geographical partition of the Web. The approach is based on the existence of multiple distributed crawlers each one responsible for the pages belonging to one or more previously identified geographical zones. The work considers a distributed crawler where the assignment of pages to visit is based on page content geographical scope. For the initial assignment of a page to a partition we use a simple heuristic that marks a page within the same scope of the hosting web server geographical location. During download, if the analyze of a page contents recommends a different geographical scope, the page is forwarded to the well-located web server. A sample of the Portuguese Web pages, extracted during the year 2005, was used to evaluate: a) page download communication times and the b) overhead of pages exchange among servers. Evaluation results permit to compare our approach to conventional hash partitioning strategies.

2005Conference object

Open access

Scalable multithreading in a low latency myrinet cluster

Publication . Alves, Albano; Pina, António; Rufino, José; Exposto, José

In this paper we present some implementation details of a programming model – pCoR – that combines primitives to launch remote processes and threads with communication over Myrinet.B asically, we present the efforts we have made to achieve high performance communication among threads of parallel/distributed applications. The expected advantages of multiple threads launched across a low latency cluster of SMP workstations are emphasized with a graphical application that manages huge maps consisting of several JPEG images.

2002Conference object

Open access

Deploying applications in multi-SAN SMP Clusters (extended version)

Publication . Alves, Albano; Pina, António; Rufino, José; Exposto, José

The effective exploitation of multi-SAN SMP clusters and the use of generic clusters to support complex information systems require new approaches; multi-SAN SMP clusters introduce new levels of parallelism and traditional environments are mainly used to run scientific computations. In this paper we present a novel approach to the exploitation of clusters that allows integrating in a unique metaphor: the representation of physical resources, the modelling of applications and the mapping of application into physical resources. The proposed abstractions favoured the development of an API that allows combining and benefiting from the shared memory, message passing and global memory paradigms.

2009Journal article

Open access

Orientação ao recurso: um modelo de comunicação para a computação em clusters

Publication . Alves, Albano; Pina, António; Rufino, José; Exposto, José

A orientação ao recurso é um novo modelo de comunicação e de computação capaz de tirar partido da comunicação de elevado desempenho, no desenvolvimento de uma classe importante de aplicações paralelas/distribuídas de grande escala. Neste contexto, a biblioteca RoCL, especialmente desenhada para a execução em ambientes cluster, de máquinas SMP, usando múltiplos fios-de-execução, permite a exploração de múltiplas tecnologias de comunicação de elevado desempenho e múltiplos protocolos de comunicação, tais como a Myrinet (GM) e a Gigabit Ethernet (VIA). Os recursos são as abstracções usadas para modelar as entidades aplicacionais que podem ser registadas e localizadas através de um serviço básico de directório, distribuído pelos nodos do cluster.

2003Conference object

Open access

pCoR - a protoype for resource oriented computing

Publication . Pina, António; Oliveira, Vitor; Moreira, Cecília; Alves, Albano

In this paper we present CoR a resource oriented computing model that address the question of how to integrate user-level fine-grained multithreading, communication and coordination into a cluster of symmetrical multiprocessor computers. To support the design of complex distributed application using the proposed paradigm we built pCoR a run-time system which has new areas that represents extensions to the strict shared memory and message passing models supported by other platforms: remote operations, dynamic domains, communication ports, multithreading management, shared memory, replication and partition are some of its distinguished features. In addition, it provides a thread-safe transport communication layer to take advantage of modern high-performance commodity hardware/software like Myrinet network.

2002Conference object

Open access