AMRA: Augmented Reality assistance in train maintenance tasks Jean-Yves Didier David Roussel Malik Mallem Samir Otmane Laboratoire Syst` emes Complexes - Universit´ e d’Evry 40, rue du Pelvoux - 91000 Evry - FRANCE {didier, roussel, mallem, otmane}@iup.univ-evry.fr Sylvie Naudet Quoc-Cuong Pham Steve Bourgeois Commissariat ` a l’ ´ Energie Atomique (CEA) - CEA/LIST/SARC CEN SACLAY - 91191 Saclay - FRANCE {sylvie.naudet, quoc-cuong.pham, steve.bourgeois}@cea.fr Christine M´ egard Christophe Leroux CEA/LIST/SCRI - CEN FAR BP 6 92265 Fontenay-aux-Roses - FRANCE {christine.megard, christophe.leroux}@cea.fr Arnaud Hocquard Alstom Transport 23-25 Avenue Morane Saulnier 92364 Meudon la Forˆ et Cedex - FRANCE arnaud.hocquard@transport.alstom.com Abstract The AMRA project, carried out by a consortium including industrials and research partners, aims at implementing an Augmented Reality (AR) system for mobile use in industrial applications such as train maintenance and repairs in in- dustrial sites. The adopted solution is a video see-through system where a tablet-PC is used as an augmented window. The overall architecture of a prototype is unfolded, and its key points are detailed. For instance, a visual registration system has been developed to accurately overlay a video stream with information. A robust, real time registration, using a single camera tied to the tablet-PC, is performed. Besides, a hierarchical description of maintenance proce- dure is set up and enriched by new media such as photos, video and/or 3D models. These 3D models have been spe- cially tailored to meet maintenance tasks requirements. The obtained multimedia contents allow easy access to techni- cal documentation through a man machine interface man- aging a multimedia engine. All these features have been combined in the AMRA prototype which have been evalu- ated by a maintenance operator. Keywords: augmented reality, registration, real-time ob- ject tracking, man-machine interface, industrial mainte- nance 1. Introduction The AMRA project, sponsored by the French Ministry of Research, is carried out by a consortium of partners includ- ing industrials and research labs. This project aimed at im- plementing an Augmented Reality system for mobile use in industrial applications, especially train maintenance. There were two objectives for such a system: the first of them was to transform cumbersome paper technical manu- als to an electronic multimedia document allowing an eas- ier data browsing. The second objective was to provide an helper application which could provide assistance and shorten the training course of a new-comer maintainer on real devices. A video see-through system was developed, based on a tablet-PC used as an augmented window on the real world. This kind of systems raises some challenging issues such as: mobile computing, real-time object/camera registration, and contextual graphic help. Since Augmented Reality involves field experimentations, an architecture has to be defined for sharing resources and data streams between mobile computer (on a tablet-PC) and remote computer(s) hosting maintenance database and heavy computing algorithms as detailed in section 3. Among the critical tasks of mobile Augmented Reality ap- pears the registration, since there is relative motion be- hal-00339457, version 1 - 17 Sep 2009 Author manuscript, published in "Workshop Industrial Augmented Reality, 4th ACM/IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2005), Vienna : Austria (2005)"