Using Real Objects to Communicate with Virtual Characters Patrick Gebhard, Martin Klesen German Research Center for Artificial Intelligence DFKI GmbH, Stuhlsatzenhausweg 3, D-66123 Saarbrücken {patrick.gebhard, martin.klesen}@dfki.de Abstract. We present an interactive installation with life-size virtual agents that inform, entertain, encourage, and assist visitors during the process of building a car. It will be installed as an exhibit in an automobile theme park. Visitors can take car elements from a shelf and put them on a workbench. The virtual agents continually comment the visitor’s actions and the current state of the construc- tion. We use Radio Frequency Identification (RFID) devices to monitor the lo- cation of the car elements. This technology allows us to design a natural, unob- trusive and robust interaction by letting the visitors using real objects to communicate with our virtual characters. We show how such an interactive presentation can be created with our SceneMaker authoring tool. We address the problem of authoring content for a large number of combinations and we explain how to design the interaction for an installation where visitors can do anything at anytime. 1 Introduction Intelligent virtual agents live – by definition – in a virtual world. The human user however lives and acts in the real world. This raises some fundamental questions. How can we bridge the gap between the real and the virtual world and how can we create common bonds between them? How do the virtual characters “know” what’s going on out there? How can we establish a kind of co-presence in a shared environ- ment? Depending on the kind of application, the available input and output devices and the target audience these questions can be addressed quite differently. The environment for our installation is a theme park where visitors can experience and interactively explore past, present, and future trends in automobile construction and design. The exhibits address different aspects like, for example, safety features and environmental issues and are used by dozens and sometimes hundreds of visitors per day. The target audience is not restricted to a specific age or user group. Installa- tions usually run 24 hours a day over a period of several weeks and sometimes month and require a minimum of supervision by members of the staff. Looking for new attractions, we have been asked to build an interactive installation with life-size virtual characters that inform, entertain, encourage, and assist the visi- tors in the task of building a car. Using a small set of front, cockpit, middle and rear elements, visitors can build different car types (a convertible, a limousine, a SUV,