R. Shumaker (Ed.): Virtual and Mixed Reality, LNCS 5622, pp. 503–510, 2009. © Springer-Verlag Berlin Heidelberg 2009 Robust Hybrid Tracking with Life-Size Avatar in Mixed Reality Environment Q.C.T. Tran, S.P. Lee, W.R. Pensyl, and D. Jernigan Interaction & Entertainment Research Centre – IERC Nanyang Technological University, Singapore {qui_tran,splee,wrpensyl,DJernigan}@ntu.edu.sg Abstract. We have developed a system which enables us to track participant- observers accurately in a large area for the purpose of immersing them in a mixed reality environment. This system is robust even under uncompromising lighting conditions. Accurate tracking of the observer’s spatial and orientation point of view is achieved by using hybrid inertial sensors and computer vision techniques. We demonstrate our results by presenting life-size, animated human avatars sitting in real chairs, in a stable and low-jitter manner. The system in- stallation allows the observers to freely walk around and navigate themselves in the environment even while still being able to see the avatars from various angles. The project installation provides and exciting way for cultural and historical narratives to be presented vividly in the real present world. 1 Objective and Significance The objective of this project is to create a low-cost, easy-to-set-up, robust and reli- able, interactive and immersive augmented and mixed reality system which presents life-size, animated human avatar in a cultural heritage environment. The significance of this project is that it provides an environment where people can interact with historical human characters in a cultural setting. Other than for cultural and historical installation, this work has the potential of being further developed for the purpose of education, art, entertainment and tourism promotion. 2 Introduction The motivation for developing this project is to allow people to experience pseudo- historical events impressed over present day real world environment (we have set our project at the famous Long Bar at the Raffles Hotel in Singapore). It is an augmented reality installation which involves re-enactment of the famous people who frequented the bar in the early 20 th century. It uses augmented reality technology to develop both historical and legendary culturally significant events into interactive mixed reality experiences. Participants wearing head-mounted display systems witness virtual char- acter versions of various notable figures, including Somerset Maugham, Joseph Conrad, and Jean Harlow, immersed within a real world environment modeled on the Raffles Hotel Long Bar they had frequented. Through the application of research in