IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 54, NO. 3, JUNE 2005 1333 Hierarchical Animation Control of Avatars in 3-D Virtual Environments Xiaoli Yang, Member, IEEE, Dorina C. Petriu, Senior Member, IEEE, Thom E. Whalen, and Emil M. Petriu, Fellow, IEEE Abstract—This paper reports the development of a three-layer hierarchical control system for the animation of anthropomorphic avatars in three-dimensional virtual environments. The lower layer controls the movement of the avatar’s joints, the second layer de- fines skills or basic behaviors (such as walk, run, and jump), and the highest layer executes a behavior-based script language that can be used to describe stories to be enacted by the avatars. In order to verify how flexible the proposed hierarchical approach is, the two lower control layers were implemented in two different ways, without affecting the script language layer. Index Terms—Animation, anthropomorphic avatar, be- havior-based control, script language, virtual environments. I. INTRODUCTION V IRTUAL environment actors are represented by icons, called avatars, which can move around and manipulate objects in the virtual environment. Currently, avatars vary from completely abstract geometric figures to rigid models that look like a hybrid between a Lego toy and a Barbie doll. Neverthe- less, there is interest for realistic anthropomorphic models for a multitude of applications such as: multimedia communications, computer graphics in entertainment, experiments in natural lan- guage interactions, interpersonal collaboration, and interfaces for avatar control. There are a variety of approaches to avatar animation that at- tempt to simulate real human beings as closely as possible in three-dimensional (3-D) virtual environments. The Virtual Re- ality Modeling Language (VRML) [6], [20], [30] is frequently used for the definition of the avatars, and more rarely used as the visualization and integration technology. Only some simple and repeatable procedures can be simulated this way, as VRML itself is not a programming language and does not give the user many options to interact with the virtual environment. Another approach is to represent avatars by real-time agents [4]. For the avatar movements, a time series of joint angles are stored so that specific motions can be replayed under real-time Manuscript received November 3, 2003; revised September 2, 2004. This work was supported in part by the Natural Sciences and Engineering Research Council (NSERC) of Canada, in part by the National Capital Institute of Telecommunications (NCIT), and in part by the Communications Research Centre (CRC) Canada. X. Yang is with the Department of Electrical Engineering, Lakehead Univer- sity, Thunder Bay, ON P7B 5E1, Canada. D. C. Petriu is with the Systems and Computer Engineering Department, Car- leton University, Ottawa, ON K1S 5B6 Canada. T. E. Whalen is with the Communications Research Center, Ottawa, ON K2H 8S2, Canada. E. M. Petriu is with the School of Information Technology and Engineering (SITE), University of Ottawa, Ottawa, ON K1N 6N5, Canada, Digital Object Identifier 10.1109/TIM.2005.847134 constraints [8]. The major advantages of prestored motions are safer algorithmic security and fast execution speed (by min- imizing the computation). However, the disadvantage is that prestored motions lack generality (since every joint must be controlled explicitly) and anthropometrical extensibility (since changing joint-to-joint distances will change the computed lo- cations of end effectors such as feet, making external constraints and contacts impossible to maintain) [4]. The Improv System developed by Perlin at New York Uni- versity mainly controls avatars by behavioral scripts [14]–[16]. The behavioral scripts are designed such that they can be trans- lated easily from a given storyboard. In this system, noise tech- niques are used to change image rendering to character move- ment. Characters created in this manner can smoothly blend and layer animations. The system is useful for predefined scenarios in the virtual environment. External functions need to be added if the approach is used in a real-time controlled virtual environ- ment and the characters need to make their decisions often. Agent Common Environment (ACE), developed by the EPFL computer Graphics Laboratory, is a platform for virtual human agent simulation that is able to coherently manage a shared vir- tual environment [10]. This system can simulate scenarios in the virtual environment. It provides built-in commands for percep- tion and acting. It also defines an external collection of behav- ioral plug-ins. The user can use these behaviors to create a story for the virtual environment, but must extend some control func- tions if (s)he wants to use this system for a specific task. Script languages are currently developed to allow complex avatar story development. The existent script languages are ap- plication dependent. One class of languages is intended to de- scribe movements with little provision for interactions and sen- sory input, [22] and [15]. Other class is intended for the de- scription of interactions, but provide little provision for story development, [5] and [9]. Since, in our case, the script will de- scribe complex stories, we will combine the features of these two classes. The “interval script language” proposed in [19] could provide the temporal dimension for the script. This paper is an extended version of the work presented by the authors in [18], [23]. It describes a three-layer hierarchical con- trol system for the animation of avatars, which are represented according to the International ISO/IEC Humanoid Standard pro- posed by the Humanoid Animation Group [25]. The lower layer controls the rotation of the avatar’s joints. The second layer de- fines skills or basic behaviors (such as walk, run, and jump), which can be combined in sequence or in parallel. The last level of abstraction is represented by a script language, which can be used to describe stories that will be enacted by the avatars. In 0018-9456/$20.00 © 2005 IEEE