Virtual Full Replication by Adaptive Segmentation * Gunnar Mathiason, Sten F. Andler University of Sk¨ ovde {gunnar.mathiason, sten.f.andler}@his.se Sang H. Son University of Virginia son@cs.virginia.edu Abstract We propose Virtual Full Replication by Adaptive segmentation (ViFuR-A), and evaluate its ability to maintain scalability in a replicated real-time database. With full replication and eventual consistency, transaction timeliness becomes independent of net- work delays for all transactions. However, full replication does not scale well, since all updates must be replicated to all nodes, also when data is needed only at a subset of the nodes. With Vir- tual Full Replication that adapts to actual data needs, resource usage can be bounded and the database can be made scalable. We propose a scheme for adaptive segmentation that detects new data needs and adapts replication. The scheme includes an architec- ture, a scalable protocol and a replicated directory service that together maintains scalability. We show that adaptive segmen- tation bounds the required storage at a significantly lower level compared to static segmentation, for a typical workload where the data needs change repeatedly. Adaptation time can be kept con- stant for the workload when there is sufficient resources. Also, the storage is constant with an increasing amount of nodes and linear with an increasing rate of change to data needs. 1 Introduction In a distributed database, performance can be improved by allo- cating data at the nodes where data is mostly used. With real-time databases, transaction timeliness is a major concern and there need to be performance guarantees. By allocating a replica of data to ev- ery node where the data might be used, transactions do not need to access data remotely over the network, and transaction time- liness becomes independent of delays on the network. In a fully replicated database the entire database is available at all nodes, but there are typically replicas that will never be used. Further, full replication uses excessive system resources, since the system must replicate all updates to all the nodes. This causes a scalability problem in the usage of bandwidth for replication of updates, us- age of storage for data replicas, and use of processing for replicat- ing updates and resolving conflicts for concurrent and conflicting updates. We have developed an adaptive approach for virtual full repli- cation (ViFuR). ViFuR manages resources for scalability, by adapting to changes in data needs during execution. ViFuR re- duces resource usage by avoiding irrelevant replication that blindly sends updates to all nodes. In this paper we extend our previous approach of static segmentation based on pre-specified data needs (ViFuR-S) [12], with adaptation to new data needs during exe- cution (ViFuR-A). A segmented database scales with the degree of required replication of data objects, and therefore it has scal- able usage of resources when the required degree of replication is bounded. The required degree of replication is application depen- dent, but for many systems it can be assumed that only a small number of replicas are required for most of the data objects. The use of a distributed database as a suitable infrastructure for communication in emergency management has been proposed by Tatomir and Rothkrantz [14]. It may be used as a whiteboard for reading and publishing current information about the state of a mission, supporting situation awareness through information col- lected cooperatively by all the users. With a whiteboard archi- tecture, users need no specific addressing of recipients within the communication infrastructure. Consider rescue teams for wildfire- fighting, equipped with portable computers that can communicate. Firefighters share mission information through a distributed real- time database. Some sub-groups of firefighters may share specific information of mutual interest. The firefighters have local dead- lines to consider, such as updates to status of open retreat paths, status of supplies, and location of colleagues. Local information may conflict with the global view of information received from command centers, and such conflicts must be detected and re- solved. The wildfire-fighting scenario highlights the need for a large-scale distributed real-time database approach. 2 Background 2.1 The DeeDS Database Architecture The main property of real-time systems is timeliness, which can only be achieved with predictable resource usage and with sufficiently efficient execution. To improve predictability, the dis- tributed real-time database system DeeDS [1] stores its database entirely in main memory, to avoid disk I/O delays caused by un- predictable access times for hard drives. Also, accesses to main memory are many times faster. To avoid transaction delays due to unpredictable network delays, the database is (virtually) fully replicated to all nodes. This makes transaction timeliness inde- pendent of network delays and network partitioning, since there is no need for remote data access during transactions. In DeeDS, transactions always execute at the local node, where all objects 1