pDomus: a prototype for Cluster-oriented Distributed Hash Tables Jos´ e Rufino * , Albano Alves, Jos´ e Exposto Polytechnic Institute of Braganc ¸a, 5300-854 Braganc ¸a, Portugal {rufino, albano, exp}@ipb.pt Ant´ onio Pina University of Minho, 4710-057 Braga, Portugal pina@di.uminho.pt Abstract The Domus architecture for Distributed Hash Tables (DHTs) is specially designed to support the concurrent de- ployment of multiple and heterogeneous DHTs, in a dy- namic shared-all cluster environment. The execution model is compatible with the simultaneous access of several dis- tributed/parallel client applications to the same or different running DHTs. Support to distributed routing and storage is dynamically configurable per node, as a function of appli- cations requirements, node base resources and the overall cluster communication, memory and storage usage. pDomus is a prototype of Domus that creates an envi- ronment where to evaluate the model embedded concepts and planned features. In this paper, we present a series of experiments conduced to obtain figures of merit i) for the performance of basic dictionary operations, and ii) for the storage overhead resulting from several storage technolo- gies. We also formulate a ranking formula that takes into account access patterns of clients to DHTs, to objectively select the most adequate storage technology, as a valuable metric for a wide range of application scenarios. Finally, we also evaluate client applications and services scalabil- ity, for a select dictionary operation. Results of the overall evaluation are promising and a motivation for further work. 1. Introduction In the field of Cluster Computing, data-intensive appli- cations may build on distributed dictionaries.A dictionary is a data repository that holds <key, data> records, which may be uniquely accessed using the key field. Dictionar- ies require support for a typical set of basic operations like insertions, retrievals, removals, membership queries, etc. * Supported by the portuguese grants PRODEP III/5.3/N/199.006/00 and FCT SAPIENS/41739/CHS/2001 Distributed Hash Tables (DHTs) are often used to im- plement distributed dictionaries. Research in DHTs started with models tailored to small/medium-scale networks of workstations (NOWs) [10, 8, 9], and continued with more recent contributions focused in large-scale/internet-wide peer-to-peer (P2P) scenarios [12, 18, 14]; the later de- mand support for i) efficient routing, ii) wide-area intermit- tent network connections, iii) continuous arrival/departure of nodes, iv) security/anonymity, etc; this requisites may be relaxed in a typical cluster, a tightly integrated hard- ware/software environment, of a much lower scale, running on private (and often very high-bandwidth) local networks. Deploying a P2P-oriented DHT platform [13, 18] in a cluster environment may thus not be advisable. When im- plementations are targeted to specific scenarios, they usu- ally include functionalities that may be useless in others (and even result in performance penalties). For instance, a distributed lookup strategy may be unappropriate when the number of nodes of a DHT is relatively small, in which case complete lookup information may be replicated at each node, or a deterministic lookup algorithm may also be used. This paper elaborates on the evaluation of pDomus, a prototype of the Domus architecture. pDomus allows to de- ploy, operate and manage multiple DHTs and its support- ing services. Management facilities include the deactiva- tion/(re)activation of DHTs and the shutdown/restart of ser- vices or even entire Domus deployments. Domus DHTs also benefit from a rich set of user-level attributes, provid- ing support for a range of i) hash functions, ii) storage plat- forms, iii) distribution constraints, iv) lookup strategies, etc. The evaluation of pDomus focused 1) on the perfor- mance of the set of storage technologies currently sup- ported, and 2) on the scalability of a class of client/server deployments. In addition, we introduce a ranking formula function to select the most appropriate storage technology. The remaining of the paper is organized as follows: sec- tion 2 revises the Domus architecture and its foundations, 15th EUROMICRO International Conference on Parallel, Distributed and Network-Based Processing (PDP'07) 0-7695-2784-1/07 $20.00 © 2007