Technologies for Augmented Reality: Calibration for Real-Time Superimposition on Rigid and Simple-Deformable Real Objects Yann Argotti 1 , Valerie Outters 2 , Larry Davis 1 , Ami Sun 2 , and Jannick P. Rolland 1,2 1 School of Electrical Engineering and Computer Science 2 School of Optics University of Central Florida 4000 Central Florida Boulevard Orlando, Florida 32816-2700, USA jannick@odalab.ucf.edu Abstract. A current challenge in augmented reality applications is the ability to superimpose synthetic objects on real objects within the envi- ronment. This challenge is heightened when the real objects are in mo- tion and/or are non-rigid. Yet even more challenging is the case when the moving real objects involved are deformable. In this article, we present a robust method for calibrating marker-based augmented reality applica- tions to allow real-time, optical superimposition of synthetic objects on dynamic rigid and simple-deformable real objects. Moreover, we illustrate this general method with the VRDA Tool, a medical education applica- tion related to the visualization of internal human knee joint anatomy on a real human knee. 1 Introduction In a large range of fields, the ability to enhance reality with synthetic informa- tion is an exciting alternative to traditional methods of acquiring information. Applications where computer-generated objects are employed to augment user perception of the real environment are referred to as augmented reality (AR) applications. A current challenge in AR applications is the ability to superimpose synthetic objects on real objects within the environment. To overcome this challenge, objects in the environment must be accurately tracked and the relationships between real and synthetic objects must be precisely determined. When dealing with medical AR applications, the real and synthetic objects in the environment are often human anatomical structures. Accurately tracking human motion is a difficult task. However, there have been attempts to understand and quantify human motion. Spoor and Veldpaus published a method for calculating rigid body motion from the spatial coor- dinates of markers that has been adapted to tracking skeletal motion [13]. In W. Niessen and M. Viergever (Eds.): MICCAI 2001, LNCS 2208, pp. 675–682, 2001. c  Springer-Verlag Berlin Heidelberg 2001