Specifying MPEG-4 body behaviors Anthony Guye-Vuill` eme, Daniel Thalmann Computer Graphics Laboratory Swiss Federal Institute of Technology CH-1015 Lausanne, Switzerland {Anthony.Guye-Vuilleme, Daniel.Thalmann}@epfl.ch Abstract The MPEG-4 standard specifies a set of low-level ani- mation parameters for body animation, but does not provide any high-level functionality for the control of avatars or em- bodied agents. In this paper, we discuss the required fea- tures for a script format allowing designers to easily spec- ify complex bodily behaviors, and describe a system and its associated syntax - Body Animation Script (BAS) - which fulfills these requirements in a flexible way. The described architecture allows the organization and parametrization of predefined MPEG-4 animations and their integration with real-time algorithmic animations, such as pointing at a spe- cific location or walking. This system has been implemented at EPFL in the framework of the EU SoNG project, in or- der to allow intelligent software agents to control their 3D graphical representation and end-users to trigger rich non- verbal behaviors from an online interface. It has been inte- grated into AML - the Avatar Markup Language. 1. Introduction Shared Virtual Environments (SVE) using avatars and other artificial body representations are very popular nowa- days. Such systems as the Blaxxun community platform[2] or Active World[1] allow users to meet at specific loca- tions, recognize each other thanks to their embodiments and interact nonverbally by triggering predefined body an- imations, in addition to the classic chat functionality. The SoNG (portalS of Next Generation) project aims at devel- oping a MPEG-4 platform for the next generation of such applications, allowing end-users to access diverse resources and services in an easy and natural way using 3D computer graphics elements, intelligent agents, new user-interfaces for rich human/agent interaction and real-time audio-visual communication. In order to demonstrate how these new technologies can be integrated, a sample e-commerce ap- plication is to be developed. Our work constitutes one of the building block aiming at al- lowing intelligent virtual sales assistants to guide the users in a rich shopping metaphor, e.g. pointing at an object while describing its characteristics, and permitting customers to interact with each other in a multimodal way, e.g. man- aging their body language. What is missing in order to al- low rich avatar and agent body behaviors, is an intermediate layer between the decision making ability of the intelligent agent and the low-level rendering capabilities of the MPEG- 4 player. Though it would be possible for the agent to di- rectly generate the low-level animation data, it would place on it a useless burden and would make difficult for content providers to finely design and control the behavioral out- come. The added value of an extra layer allowing the reuse of predefined animations by providing parametrization fa- cilities is also to be taken into account. Body animation in MPEG4 is defined in terms of 296 BAPs - Body Animation Parameters, which represent a cer- tain Degree Of Freedom (DOF) of an anatomical articula- tion. DOF can be translations or rotations along an axis. These BAPs can be applied to any MPEG-4 compliant body and produce the same animation. 2. Previous work Current shared virtual environments (e.g. [2][1][4]) provide very limited functions, typically predefined ges- tures, e.g. bow, which are directly triggered by a GUI or keyboard command. More advanced system relying on rule-based techniques, like IMPROV[10] or BEAT and BodyChat[3][13], have produced interesting results but they do not allow a clear separation between the purely deliber- ating function and the behavior specification itself. Using different approaches, the Motivate system[9] and ACE[8] also tackle the issue of behavior specification but have not been designed to be used in MPEG-4 systems.