Designing, Developing, and Evaluating Context-Aware Systems * Yoosoo Oh 1 , Albrecht Schmidt 2 , and Woontack Woo 1 1 GIST U-VR Lab. Gwangju 500-712, S.Korea {yoh, wwoo}@gist.ac.kr 2 Fraunhofer IAIS and b-it, University of Bonn, Schloss Birlinghoven, S.t Augustin, Germany albrecht.schmidt@iais.fraunhofer.de * This work was supported by Seondo project of MIC, Korea. Abstract Context and context-awareness have been central issues in ubiquitous computing research for the last decade. Advances with regard to context acquisition and activity recognition allow interesting small scale applications. However in larger systems including many sensors and actuators and spanning multiple administrative domains are still remain as unsolved central issues. Particularly, in the areas of reasoning and context-fusion there are many open questions. In this paper we motivate large scale context-aware systems that have support for the full life cycle using a prototype that was implemented and tested. We then describe in detail the underlying architecture which supports dynamic context-aware systems and includes mechanisms for context fusion and reasoning. Also, we propose a new approach for evaluating context-aware systems. The approach is an adapted expert evaluation; well known in the user interface domain, but using a carefully selected set of heuristics specifically targeted at context-aware systems. 1. Introduction Context-aware systems have been a research focus in ubiquitous computing since its very beginning [12]. Many systems have been developed in the research community using various sensors and acting on different contexts. However outside the research laboratories, context-aware systems have had little impact, with the exception of location aware applications. Location-aware applications (e.g. in car navigation) and location based services prove the usefulness of the overall concept as the first real commercial context-aware applications. Nevertheless, many issues with regard to more generic context-aware systems beyond laboratory scale are still unsolved. In particular the life-cycle support of context-aware systems (simulation, installation, debugging, and maintenance, adding new entities, removing old entities, and upgrading system components) has received little attention. These issues need to be addressed on an architectural level to provide generic solutions. In this paper we will introduce a new and open architecture for context-aware systems that provides mechanisms to solve these problems in parts. Advances in context-aware computing have been published in the area of acquiring activity and context information using various sensors [2], [15]. This research indicates that for given situations it is possible to create sensors and recognize subsystems that provide context information to a larger system. Looking at the development cycle, it is of interest to support the simulation of context recognition. As indicated in recent work on prototyping [3], having manually simulated sensors can ease the early phases of development of context-aware systems. Similarly improvement in sensors and recognition algorithms are common and hence a system has to provide for changes on this level. When designing and developing context-aware systems and architectures, one central issue is to show that the presented solution is appropriate and that it fulfills the criteria set. In the literature, several approaches can be found that evaluate the suitability of a solution. Often the argument is made that the architecture was successfully applied to one or more projects, e.g. [1]. In our view, this is not necessarily a good measure of the quality as it is usually not clear how much a specific solution relies on the presented architecture and where the clear benefits in the development and deployment were. Another common approach is that researchers compare their approach to 2007 International Conference on Multimedia and Ubiquitous Engineering(MUE'07) 0-7695-2777-9/07 $20.00 © 2007