Augmenting Gesture Animation with Motion Capture Data to Provide Full-Body Engagement Pengcheng Luo 1 , Michael Kipp 2 and Michael Neff 1 1 UC Davis, USA, {pcluo|neff}@ucdavis.edu 2 DFKI, Germany, michael.kipp@dfki.de Abstract. Effective speakers engage their whole body when they ges- ture. It is difficult, however, to create such full body motion in animated agents while still supporting a large and flexible gesture set. This pa- per presents a hybrid system that combines motion capture data with a procedural animation system for arm gestures. Procedural approaches are well suited to supporting a large and easily modified set of gestures, but are less adept at producing subtle, full body movement. Our sys- tem aligns small motion capture samples of lower body movement, and procedurally generated spine rotation, with gesture strokes to create con- vincing full-body movement. A combined prediction model based on a Markov model and association rules is used to select these clips. Given basic information on the stroke, the system is fully automatic. A user study compares three cases: the model turned off, and two variants of our algorithm. Both versions of the model were shown to be preferable to no model and guidance is given on which variant is preferable. Key words: Embodied Conversational Agents, Posture Synthesis, Mo- tion Capture 1 Introduction When creating virtual agents, the designer is caught between two main animation options, each with their inherent trade-offs. Procedural motion generation offers excellent control, allowing the agent to flexibly respond to a range of situations and generate a very large set of gestures. This flexibility, however, comes at the cost of extra work and/or realism as it is difficult to generate highly realistic motion using procedural methods. On the other hand, motion capture-based approaches provide an easier method to obtain realistic motion that engages the entire body, but control is generally limited. In this paper, we present a hybrid system that uses procedural generation for arm gestures and motion capture data to add realistic body movement 3 . Our pro- cedural methods for arm gesture are based on previously published techniques [1, 2]. The contribution of this work is a system for engaging the rest of the body, 3 Animation samples can be found on http://www.cs.ucdavis.edu/ ~ neff/ pengcheng/