Loading...
21 results
Search Results
Now showing 1 - 10 of 21
- Full-speed scalability of the pDomus platform for DHTsPublication . Rufino, José; Pina, António; Alves, Albano; Exposto, José; Lopes, Rui PedroDomus is an architecture for Distributed Hash Tables (DHTs) tailored to a shared-all cluster environment. Domus DHTs build on a (dynamic) set of cluster nodes; each node may perform routing and/or storage tasks, for one or more DHTs, as a function of the node base (static) resources and of its (dynamic) state. Domus DHTs also benefit from a rich set of user-level attributes and operations. pDomus is a prototype of Domus that creates an environment where to evaluate the architecture concepts and features. In this paper, we present a set of experiments conduced to obtain figures of merit on the scalability of a specific DHT operation, with several lookup methods and storage technologies. The evaluation also involves a comparison with a database and a P2P-oriented DHT platform. The results are promising, and a motivation for further work.
- pDomus: a prototype for cluster-oriented distributed hash tablesPublication . Rufino, José; Pina, António; Alves, Albano; Exposto, JoséThe Domus architecture for distributed hash tables (DHTs) is specially designed to support the concurrent deployment of multiple and heterogeneous DHTs, in a dynamic shared-all cluster environment. The execution model is compatible with the simultaneous access of several distributed/parallel client applications to the same or different running DHTs. Support to distributed routing and storage is dynamically configurable per node, as a function of applications requirements, node base resources and the overall cluster communication, memory and storage usage. pDomus is a prototype of Domus that creates an environment where to evaluate the model embedded concepts and planned features. In this paper, we present a series of experiments conduced to obtain figures of merit i) for the performance of basic dictionary operations, and ii) for the storage overhead resulting from several storage technologies. We also formulate a ranking formula that takes into account access patterns of clients to DHTs, to objectively select the most adequate storage technology, as a valuable metric for a wide range of application scenarios. Finally, we also evaluate client applications and services scalability, for a select dictionary operation. Results of the overall evaluation are promising and a motivation for further work
- Deploying applications in Multi-SAN SMP clustersPublication . Alves, Albano; Pina, António; Rufino, José; Exposto, JoséThe effective exploitation of multi-SAN SMP clusters and the use of generic clusters to support complex information systems require new approaches. On the one hand, multi-SAN SMP clusters introduce another level of parallelism which is not addressed by conventional programming models that assume a homogeneous cluster. On the other hand, traditional parallel programming environments are mainly used to run scientific computations, using all available resources, and therefore applications made of multiple components, sharing cluster resources or being restricted to a particular cluster partition, are not supported. We present an approach to integrate the representation of physical resources, the modelling of applications and the mapping of application into physical resources. The abstractions we propose allow to combine shared memory, message passing and global memory paradigms.
- Orientação ao recurso: um modelo de comunicação para a computação em clustersPublication . Alves, Albano; Pina, António; Rufino, José; Exposto, JoséA orientação ao recurso é um novo modelo de comunicação e de computação capaz de tirar partido da comunicação de elevado desempenho, no desenvolvimento de uma classe importante de aplicações paralelas/distribuídas de grande escala. Neste contexto, a biblioteca RoCL, especialmente desenhada para a execução em ambientes cluster, de máquinas SMP, usando múltiplos fios-de-execução, permite a exploração de múltiplas tecnologias de comunicação de elevado desempenho e múltiplos protocolos de comunicação, tais como a Myrinet (GM) e a Gigabit Ethernet (VIA). Os recursos são as abstracções usadas para modelar as entidades aplicacionais que podem ser registadas e localizadas através de um serviço básico de directório, distribuído pelos nodos do cluster.
- A cluster oriented model for dynamically balanced DHTsPublication . Rufino, José; Pina, António; Alves, Albano; Exposto, JoséIn this paper, we refine previous work on a model for a Distributed Hash Table (DHT) with support to dynamic balancement across a set of heterogeneous cluster nodes. We present new high-level entities, invariants and algorithms developed to increase the level of parallelism and globally reduce memory utilization. In opposition to a global distribution mechanism, that relies on complete knowledge about the current distribution of the hash table, we adopt a local approach, based on the division of the DHT into separated regions, that possess only partial knowledge of the global hash table. Simulation results confirm the hypothesis that the increasing of parallelism has as counterpart the degradation of the quality of the balancement achieved with the global approach. However, when compared with Consistent Hashing and our global approach, the same results clarify the relative merits of the extension, showing that, when properly parameterized, the model is still competitive, both in terms of the quality of the distribution and scalability.
- Um modelo cooperativo e distribuído para a recuperação de informação na WWWPublication . Exposto, José; Pina, António; Macedo, Joaquim; Alves, Albano; Rufino, JoséEste artigo apresenta um ambiente da recuperação de informação - SIRe - inteiramente adequado a um espaço multilingue e dinâmico da informação como a Internet. A arquitectura proposta visa assegurar que o sistema pode ser ampliado, para atingir um desempenho mais elevado e melhores resultados das pesquisas, ou reduzido para permitir economizar no sentido de obter uma melhor relação global para o custo/desempenho. Para atingir o almejado desempenho, como uma alternativa de baixo custo as máquinas paralelas tradicionais, o sistema proposto assenta em tecnologias de conveniência para a criação de uma arquitectura de cluster baseada em estações de trabalho multi-processadores, ligadas por infraestruturas de rede de elevado desempenho.
- Relações entre a topologia de rede e a geografia dos servidores WWW em PortugalPublication . Exposto, José; Pina, António; Macedo, Joaquim; Alves, Albano; Rufino, JoséEsta comunicação centra-se no estudo das localizações geográficas e da topologia de rede da WWW portuguesa, baseado em recolhas de rotas de encaminhamento de rede. O objectivo é a identificação das nuvens de densidade entre os servidores WWW, utilizando, para tal: medidas de proximidade temporal baseadas nos tempos médios de ida e volta (RTT), de forma a determinar as localizações mais apropriadas para a instalação de robôs cooperativos que minimizem o tempo de descarga global das fontes de informação. O processo é realizado através do recurso a técnicas vulgares de aglomeração num ambiente experimental, recorrendo a dados recolhidos, activamente, através de sondas colocadas na Internet, para criar uma grafo de distâncias fim-a-fim e calcular a distância entre os arcos dados pelos os tempos médios de ida e volta de uma comunicação de Internet entre quaisquer dois servidores. Um outro objectivo, é a confirmação da existência de relação entre a distância lógica (RTT) e a distância física (Geográfica). Os resultados dos experimentos realizados vêm confirmar as hipóteses iniciais pelo que estão abertos caminhos para novas experiências no âmbito do estudo entre as relações entre as topologias de rede e Internet e a geografia dos servidores.
- Geographical partition for distributed web crawlingPublication . Exposto, José; Macedo, Joaquim; Pina, António; Alves, Albano; Rufino, JoséThis paper evaluates scalable distributed crawling by means of the geographical partition of the Web. The approach is based on the existence of multiple distributed crawlers each one responsible for the pages belonging to one or more previously identified geographical zones. The work considers a distributed crawler where the assignment of pages to visit is based on page content geographical scope. For the initial assignment of a page to a partition we use a simple heuristic that marks a page within the same scope of the hosting web server geographical location. During download, if the analyze of a page contents recommends a different geographical scope, the page is forwarded to the well-located web server. A sample of the Portuguese Web pages, extracted during the year 2005, was used to evaluate: a) page download communication times and the b) overhead of pages exchange among servers. Evaluation results permit to compare our approach to conventional hash partitioning strategies.
- ToCL: a thread oriented communication library to interface VIA and GM protocolsPublication . Alves, Albano; Pina, António; Rufino, José; Exposto, JoséIn this paper we present ToCL a thread oriented communication library specially designed to fully exploit multithreading in a multi-networked cluster environment. ToCL provides a basic set of primitives to handle zero-copy message passing between application threads spread among cluster nodes. Large messages are fragmented and sent to remote threads as single messages using multiple low-level communication subsystems. The current implementation supports both Myrinet through GM and Gigabit Ethernet through VIA but we plan to extend it to other communication subsystems.
- Distributed paged Hash tablesPublication . Rufino, José; Pina, António; Alves, Albano; Exposto, JoséIn this paper we present the design and implementation of DPH, a storage layer for cluster environments. DPH is a Distributed Data Structure (DDS) based on the distribution of a paged hash table. It combines main memory with file system resources across the cluster in order to implement a distributed dictionary that can be used for the storage of very large data sets with key based addressing techniques. The DPH storage layer is supported by a collection of cluster-aware utilities and services. Access to the DPH interface is provided by a user-level API. A preliminary performance evaluation shows promising results.
- «
- 1 (current)
- 2
- 3
- »