SHAPE-FROM-SHADING: COMBINING PROBABILISTIC & STATISTICAL MODELS ABSTRACT In this work, we combine a probabilistic model of surface normals from shape-from-shading[1] with a statistical model of 3D shape[2]. We sample the Fisher Bingham FB8 distribution of surface normals from probabilistic SfS model using Gibbs sampling. We get surface normals from the statistical shape model. We fit individual normal distributions to each of these normals and combine them to give a product normal distribution i.e. a better model for SfS. Ahmad T., Wilson R.C., Department of Computer Science, University of York, UK {touqeer,wilson}@cs.york.ac.uk INTRODUCTION ‘To get 3D shape from a single 2D gray scale image’ i.e. Shape-from-Shading as it is commonly known in Computer Vision community; has been a major research topic for past four decades. In the last forty years a number of approaches [4] have been proposed to improve the solution. Recently an approach [1] based on directional statistics and belief propagation has been presented that have shown better results as compared to other currently available algorithms [2-4]. However Smith’s work [2] has shown great improvement over geometrical SfS algorithm [3] when a 3D statistical model of human face shapes was used. Inspired by the improvements shown by incorporation of 3D statistical model we are going to combine the probabilistic model of SFS [1] with this 3D shape model based on the Yale-B data base [2]. EXPERIMENT DETAILS PROBABILISTIC SFS Lambertian shading equation applies; The normal direction has two degrees of freedom i.e. elevation angle and the azimuth angle. The elevation angle is given by the cone constraint. This leaves one degree of freedom per pixel. The algorithm takes a conventional approach and uses smoothness as further source of information. There were two reasons to choose the FB8 distribution – firstly the multiplication of two FB8 distributions gives another FB8 distribution and secondly the sub-model of Bingham-Mardia distribution is used to represent the cone constraint. Each pixel has a prior expressing the cone constraint and the compatibility between the adjacent pixels provide the smoothness term. A graphical model i.e. a pair- wise Markov random field on a grid is constru- cted in which each node is a random variable that represents the unknown surface orientation of a pixel. Sum-product belief propagation is used to determine the marginal distribution for each node i.e. a FB8 distribution. STATISTICAL SHAPE MODEL If v is a vector on the tangent plane to S² at n S² and v 0 the exponential map denoted by of v is the point on S² along the geodesic in the direction of v at distance ||v|| from n. The point on S² is denoted by . The inverse of the exponential map is the log map denoted by . The log and exponential maps are given by, Intrinsic mean of set of points on spherical manifold is a point that minimizes the Riemann- ian distance to each point in the given set. This minimization problem is solved iteratively by gradient descent method of Pennec [6]. 2 n TS n Exp () n Exp v n Log SAMPLING of FB8 We use the slice sampling algorithm proposed by A.Kume & S.G.Walker [5] to sample from the Fisher Bingham distribution. The Fisher Bingh- am distribution is the multivariate normal distribution having a constraint to lie on the surface of a sphere. If x={x0, x1, x2} is distribut- ed according to FB distribution then | x|=1. Due to this constrain x²={x0², x1², x2²} lies on the simplex; so x is transformed to the (w, s) and a joint density of (w, s) is found using Gibbs method by introducing latent variables ( u, v, w). Here, si=xi² and wi=xi/|xi|. When transformed from x to (w0, w1, w2, s1, s2) we get the joint density of interest. The three latent variables ( u, v, w) are introduced to give the joint density as, The three latent variables are uniform and easy to sample from; the conditional densities of w and s are then found accordingly. The sets A, Au, Aw and Av are formed by inverting the inequalities involving latent variables. We calculate the intrinsic mean for each pixel location using 250 pre-aligned face shapes using the method described above. We use five iterations to get the intrinsic mean. The covariance matrix Σ1 for each pixel location is also calculated from a matrix of 250 x 3 surface normals for that particular pixel location. We define a normal distribution for each pixel location(x, y) by Ω1[P1μ1, P1 ] where P1=Σ1 is inverse of the covariance matrix μ1 is the intrinsic mean. From SfS we get Fisher Bingh- am distributions for each pixel, we apply the Gibbs sampling algorithm described above and get samples from each distribution. Through five iterations and 250 samples per iteration we get the intrinsic mean and covariance matrix for each pixel location (x, y) and fit a normal distribution Ω2[P2μ2, P2 ]. The two distributions are then multiplied to give the most probable surface normal μ* at pixel location (x, y). ΩT[P* μ* , P *]=Ω1[P1μ1, P1 ] Ω2[P2μ2, P2 ] ΩT[P * μ* , P *]=Ω[P1μ1 + P2μ2, P1 + P2 ] μ* = (P*) (P*μ*) We use Fisher Criterion to identify the outliers and zero out these surface normals in the tang- ent plane. The resultant surface normals are then fed to the Smith’s algorithm [2] of height reconstruction which uses the average height of 200 range images to give the surface height. 1 1 RESULTS Preliminary results have shown improvement over the SfS model even when the surface normals are combined through normal distributions. In future we will try to fit some more suitable distributions and iterate the method to get better results. Acknowledgement Touqeer Ahmad is thankful to EURECA scholarship programme for fully funding his M.Sc. in Computer Science(by research) at University of York. REFERENCES [1] Tom S.F. Haines, Integrating Shape-from-Shading & Stereopsis, Ph.D. Dissertation, University of York, York, UK, Sep. 2009. [2] William A.P. Smith, Statistical Methods for Facial Shape-from-Shading & Recognition, Ph.D. Dissertation, University of York, York, UK, Feb. 2007. [3] P. L. Worthington and E. R. Hancok: New Constraints on Data-Closeness and Needle Map Consistency for Shape-from-Shading. IEEE Trans. Pattern Analysis and Machine Intell., 21(12):1250-1267, 1999. [4] R. Zhang, P.S. Tsai, J. E. Cryer, and M. Shah: Shape-from-shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell., 21(8):690–706, 1999. [5] A. Kume and S.G. Walker: On the Fisher-Bingham distribution. Statistic and Computing, 19(2):167-172, 2009. [6] X. Pennec: Probabilities and statistics on Riemannian manifolds: basic tools for geometric measurements. In Proc. IEEE Workshop on Nonlinear Signal and Image Processing, 1999. Image SfS Needle Map Combined Needle Map Re-Illuminated