Animating Human Face under Arbitrary Illumination Tongbo Chen Baocai Yin Wanjun Huang Dehui Kong College of Computer Science Beijing Polytechnic University, Beijing 100022, China yinbc@bjpu.edu.cn Abstract In this paper, we present a method to animate human face under arbitrary lighting condition. We first acquire the reflectance models of a human face in various expressions by employing a robot to precisely control light position. By developing a relighting tool, named LUXMASTER, we achieve the great freedom to change the illumination con- dition and get a variety of photorealistic novel renderings of the human face. Then we introduce morphing techniques to add the time dimension and produce a clip of facial an- imation with varying illumination and expression. In order to make the morphing work easy, we design an expressive, intuitive and efficient tool, called FLUIDMAN , which at the same time enables enormous possibility of visual effects. 1. Introduction Facial animation is always an interesting challenge for re- searchers or practitioners in its nearly three decades as an identifiable area of computer graphics [21, 20]. Imaginative applications of animated graphical faces are found in so- phisticated human-computer interfaces, interactive games, multimedia titles, VR telepresence experiences, education and as always, in a broad variety of production anima- tions. Some of the most exciting applications of facial an- imation will be for independent filmmakers, as soon it will be possible for a small team of talented people to create a movie with all the visual richness of Star Wars, Titanic, or Lawrence of Arabia, without spending hundreds of millions of dollars. Indeed, a grand challenge in facial animation is the synthesis of artificial faces that look and act like real person. The solution to this challenge will involve not just computer graphics, but also other scientific disciplines such as psychology and artificial intelligence. Graphics tech- nologies underlying facial animation now run the gamut from keyframing to image morphing, video tracking, ge- ometric and physical modeling, and behavioral animation. The main goal of these technologies is to create tools that will help an animator to create realistic and aesthetically rich facial animations. When confronting the synthesis of realistic faces, it is also of paramount importance to ade- quately model auxiliary structures, such as the mouth, eyes, eyelids, teeth, lips, ears, and the articulate neck, each a non- trivial task. Sophisticated biomechanical models for realis- tic facial animation are computationally expensive. They can tax the abilities of even the most powerful graphics computers currently available. Recent endeavors have provided solutions to the prob- lems of facial animation and produced some attractive ex- amples in terms of realism. 3D photography techniques can acquire accurate geometric models of individual faces. [2, 12, 14, 23, 22] animated faces through morphing. [27] produced performance-driven animation, while [17] mod- eled and animated faces from video. Physics-based sim- ulation techniques [15, 26, 31] have helped animate the complex deformations of a face in its different expression. Video Rewrite [3] used existing footage to create automat- ically new video of a person mouthing words that she did not speak in the original footage. By stitching the warped triphone videos into the background sequence, it generated realistic lip-synched video. More recent work represents a new avenue to captur- ing the spatially varying reflectance characteristics of hu- man face. In [18], Marschner et al. described a system for modeling, animating, and rendering a face using mea- sured data for geometry, motion, and reflectance (BRDF ) – Lafortune et al. [13] introduced a general and efficient representation for the bidirectional reflectance distribution functions, or BRDFs. This system proposed an important avenue leading to rendering human face under any illumina- tion and viewing conditions. However, it is labor-intensive and requires expensive and complicated equipments, such as laser range scanner and devices for measuring albedo maps. With these constraints, it is very hard to measure efficiently the geometry or BRDF and to add auxiliary fea- tures, namely ears, eyes, hair and eyebrow. Debevec et al. [9] proposed a method to acquire the reflectance field of a human face and used these measurements to render the face