Marc-Olivier Killijian · Nicolas Rivi` ere · Matthieu Roy Experimental Evaluation of Resilience for Ubiquitous Mobile Systems Received: date / Accepted: date Abstract In this position paper, we address the needs and motivations for practical evaluation of ubiquitous mobile systems. We are particularly interested in the val- idation of resilience mechanisms for such systems. We advocate the use of real experiments that complement simulation for the evaluation of actual prototypes. In- deed, experimental validation is particularly appealing when resilience properties such as safety or availability are an issue. We then propose and discuss some techno- logical trails for the implementation of such an evalua- tion platform and its validation. Keywords Experimental evaluation · Validation · Mobile systems · Ubiquitous computing · Resilience 1 Introduction Weiser’s vision of ubiquitous computing [Wei99] is that of a society where technology has merged with the physical environment, and where systems have become ubiqui- tous, offering services to users while remaining invisible. This stealthiness implies two important aspects. First, it assumes excellent interfaces and a rigorous design. Sec- ond, it requires a level of trustworthiness such that users can forget the very presence of the technology, under any conditions and threats. To allow ubiquitous systems to achieve that level of trustworthiness, resilience mecha- nisms aimed at improving their security, reliability and availability must be devised. Mobility has a significant impact on resilience of ubiq- uitous computing. Mobility of the physical devices im- This work was partially supported by the Hidenets project (EU-IST-FP6-26979), and the ReSIST network of excellence (EU-IST-FP6-26764). M.-O. Killijian, N. Rivi` ere, M. Roy LAAS-CNRS Universit´ e de Toulouse 7, avenue du Colonel Roche 31077 Toulouse Cedex 4, France E-mail: marco.killijian@laas.fr plies open and highly dynamic environments, stressing usage conditions with multiple interactions and multiple sources of failures. Thus, new risks stem from mobility, along with new threats which can be both accidental and malicious. Indeed, risk of loss, theft or physical damage is exacerbated by mobility, but risks of energy (and other resources) exhaustion or dangerous encounters (such as malicious service discovery and usage) are new to mobile ubiquitous systems. Nevertheless, positive impacts of mobility and ubiq- uity ought to be taken into account as well. For instance, heterogeneity of the services available in a ubiquitous system can be regarded as a source of diversity that resilience mechanisms can benefit from. Using specific fault-tolerance mechanisms such as triple modular repli- cation with voting, a system can benefit from diversity to tolerate common mode failures. Without diversity, i.e. with equivalent replicas of a service, a single fault might lead the various replicas to fail in the same manner, in a common mode. Taking diversity into account, and using an appropriate mechanism, this risk is reduced. Our main goal is the study of both positive and neg- ative impacts of mobility and ubiquity on system re- silience. To this respect, we believe that it is essential to practically validate and experimentally evaluate the proposed solutions for increasing the resilience of mobile ubiquitous systems. 2 Evaluation of Resilient Ubiquitous Mobile Systems To the best of our knowledge, little research has been done on the evaluation of ubiquitous systems. Most of the literature in this domain concerns evaluation of users experience and human-computer interfaces [SK01]. How- ever, some work is also looking at defining appropri- ate metrics for the evaluation of distributed applications running on ubiquitous systems [BKL01, CCKM01]. [BR01] is looking at a general approach to evaluat- ing ubiquitous systems. In the paper, the authors ar-