Recovering Facial Shape and Albedo using a Statistical Model of Surface Normal Direction William A.P. Smith and Edwin R. Hancock Department of Computer Science, The University of York {wsmith, erh}@cs.york.ac.uk Abstract This paper describes how facial shape can be modelled using a statistical model that captures variations in sur- face normal direction. To construct this model we make use of the azimuthal equidistant projection to map surface normals from the unit sphere to points on a local tangent plane. The variations in surface normal direction are cap- tured using the covariance matrix for the projected point positions. This allows us to model variations in face shape using a standard point distribution model. We train the model on fields of surface normals extracted from range data and show how to fit the model to intensity data us- ing constraints on the surface normal direction provided by Lambert’s law. We demonstrate that this process yields ac- curate facial shape recovery and allows an estimate of the albedo map to be made from single, real world face images. 1. Introduction Shape-from-shading provides an alluring yet somewhat elusive route to recovering 3D surface shape from single 2D intensity images [15]. Unfortunately, the method has proved ineffective in recovering realistic 3D face shape be- cause of real world albedo variations and local convexity- concavity instability due to the bas-relief ambiguity. This is of course a well known effect which is responsible for a number of illusions, including Gregory’s famous inverted mask [6]. The main problem is that the nose becomes im- ploded and the cheeks exaggerated [2]. It is for this reason that methods such as photometric stereo [5] have proven more effective. One way of overcoming this problem with single view shape-from-shading is to use domain specific constraints. Several authors [1, 2, 8, 9, 16] have shown that, at the ex- pense of generality, the accuracy of recovered shape in- formation can be greatly enhanced by restricting a shape- from-shading algorithm to a particular class of objects. For instance, both Prados and Faugeras [8] and Castelan and Hancock [2] use the location of singular points to enforce convexity on the recovered surface. Zhao and Chellappa [16], on the other hand, have introduced a geometric con- straint which exploited the approximate bilateral symmetry of faces. This ‘symmetric shape-from-shading’ was used to correct for variation in illumination. They employed the technique for recognition by synthesis. However, the recovered surfaces were of insufficient quality to synthe- sise novel viewpoints. Atick et al. [1] proposed a statisti- cal shape-from-shading framework based on a low dimen- sional parameterisation of facial surfaces. Principal com- ponents analysis (PCA) was used to derive a set of ‘eigen- heads’ which compactly captures 3D facial shape. Unfortu- nately, it is surface orientation and not depth which is con- veyed by image intensity. Therefore, fitting the model to an image equates to a computationally expensive parame- ter search which attempts to minimise the error between the rendered surface and the observed intensity. This is sim- ilar to the approach adopted by Samaras and Metaxas [9] who incorporate reflectance constraints derived from shape- from-shading into a deformable model. Previous work has shown that both images of faces [13] and facial surfaces [1] can be modelled in a low- dimensional space, derived by applying PCA to a training set of images or surfaces. Unfortunately, the construction of a statistical model for the distribution of facial needle-maps is not a straightforward task. The statistical representation of directional data has proved to be considerably more dif- ficult than that for Cartesian data [7]. Surface normals can be viewed as residing on a unit sphere and may be specified in terms of the elevation and azimuth angles. This repre- sentation makes the computation of distance difficult. For instance, if we consider a short walk across one of the poles of the unit sphere, then although the distance traversed is small, the change in azimuth angle is large. Hence, con- structing a statistical model that can capture the statistical distribution of directional data is not a straightforward task. To overcome the problem, in this paper we draw on ideas from cartography. Our starting point is the azimuthal equidistant or Postel projection [12]. This projection has the important property that it preserves the distances between