An Automata-based Formalism for Cooperative Augmented Reality Systems Felix Hamza- Lup School of Computing Armstrong Atlantic State University Savannah, GA, USA felix@cs.armstrong.edu Ferucio Laurent ¸iu T ¸ iplea ∗ University of Central Florida School of Computer Science Orlando, Florida 32816, USA, tiplea@cs.ucf.edu Abstract The aim of the paper is to propose a formal model for Cooperative Augmented Reality Systems (CARSs). We motivate it by two examples, one of them applying Aug- mented Reality (AR) paradigms to medical training, and the other one to telerobotic manipulation. The model is based on automata theory and objectives are formulated as reachability-like decision problems. We show that reacha- bility, which plays an important role in analyzing CARSs, is undecidable in general, but it is NP-complete for finite- domain CARSs. The relationship with Petri nets, as models of distributed and concurrent systems, is also provided. 1 Introduction Augmented Reality (AR) systems [11] use computers and specific visualization devices to overlay virtual infor- mation in the real world. They enhance the perception of, and the interaction with the real world. Visually, the real scene a person sees is augmented with computer-generated objects. These virtual objects are placed (registered) in the real scene in such a way that the information they carry ap- pears in the correct location with respect to the real objects they augment. Several AR systems were proposed in the mid ’90s as tools to assist in different fields such as medicine [5], com- plex assembly labeling [4], and construction labeling [18]. With advances in computer graphics, networking, and hard- ware (i.e., 3D displays, haptic devices etc.) the research community has shifted attention to distributed environments that use extensively the AR paradigm [1, 14]. Further- more, a Cooperative Augmented Reality System (CARS) can substantially facilitate experts’ interactions, especially during quick-response conditions such as medical emergen- cies [15], and has the potential to provide efficient training. ∗ On leave from “Al.I.Cuza” University of Ias ¸i, Department of Com- puter Science, Ias ¸i, Romania, e-mail: fltiplea@mail.dntis.ro An important aspect regarding these systems is that aug- mentation can occur for multiple sensory modalities (hap- tic, visual, auditive). The main challenge encountered in designing CARSs is the dynamic nature of the environment. The attributes of the virtual components of the scene are changing as an ef- fect of the participants’ interactions. These interactions and information exchanges generate a state referred to as the dynamic shared state [16] that has to be maintained con- sistent at all sites for all participants and in the presence of inevitable network latency and jitter. In spite of the fact that several problems inherent in such environments have been investigated over the years, no generally accepted formal- ism allowing an in-depth reasoning for such systems has been developed. The aim of this paper is to propose a formal model for CARSs. We motivate it by two examples, one of them applying AR paradigms to medical training, and the other one to telerobotic manipulation. The model is based on au- tomata theory and objectives are formulated as reachability- like decision problems. We show that reachability, which plays an important role in analyzing CARSs, is undecidable in general, but is NP-complete for finite-domain CARSs. A relationship with Petri nets, as models of distributed and concurrent systems, is also provided. The paper is organized into six sections. Section 2 de- scribes two complex CARSs as motivating examples for this work. In Section 3 we raise the abstraction level by intro- ducing the main components of a formal model intended to capture the behavior of CARS. In Section 4, the model is applied to the first CARS example, an AR-based Endotra- cheal Intubation training system. A few basic properties of our model are studied in Section 5. We end the paper with conclusions followed by near future work. 1