A faster MRF optimization based method for Shape from Shading using Gibbs sampling with quadruplet cliques. Markus Louw Fred Nicolls Department of Electrical Engineering University of Cape Town, South Africa markus.louw@gmail.com Abstract This paper extends the MRF formulation approach developed in [7] and [6] for solving the shape from shading problem. Our method extends the Gibbs sampling approach to solve an MRF formulation which characterizes the Shape from Shading (SFS) problem under Lambertian reflectance conditions (the al- gorithm is extensible to other lighting models). Our method uses a simpler set of energy functions (on point quadruplets), which is faster to converge, but less accurate. 1. Introduction and Literature Review Two surveys, [12] (1999), and [5] (2004) describe the history of Shape from Shading algorithms. In the former, SFS approaches are classified into minimization (e.g. [10]), propagation (e.g. [8]), local (e.g. [9]), or linear (e.g. [11]) approaches. In the latter, they are classified into methods based on partial differ- ential equations (characteristic strips [1], power series expan- sion [2], and viscosity solutions (e.g.[3])), minimization meth- ods [4], and methods which approximate the image irradiance equation, which contain the local and linear methods surveyed in [12]. This work builds on that of [6], called Gibbs Multi-Scale Projective Multi-Res SFS with Occlusion handling (GMPM- SFS), in which a Markov Random Field formulation for the labels of points on a lattice is developed. In that case we min- imize a set of energy terms which correspond to differences in the synthetic reflectance map vs. observed data, with additional possibility for putting smoothness constraints on that surface. In this paper we change the energy function which is min- imized by increasing the clique sizes of the Markov Random Field. This approach requires us to treat the observed re- flectance map data (image) as if each pixel were the reflectance of light off a single plane through the four corner vertex nodes about the pixel. Using Gibbs sampling means there is a com- putational speed increase (if we used LBP, it would take much longer). This algorithm is called Gibbs Multi-Scale Projective Multi-Res with clique quadruplets SFS, or GMPM4-SFS. 2. Lambertian Reflectance Model This algorithm calculates a surface on the Lambertian assump- tion that the intensity of a pixel is proportional to the inner prod- uct of the direction vector of the incident light and the surface normal at the point of intersection. We follow the notation of [5], to formulate this. The image irradiance equation is R( - → n (x)) = I (x) (1) where I (x) is the image irradiance (usually the intensity) mea- sured at location x, and R( - → n (x)) is the reflectance function on Figure 1: On the left is shown the original MRF topology w.r.t. energy terms over cliques of corner vertex node triplets. On the right is shown our new energy term, associated with the quadru- plet and represented by a square. It is connected by lines to their corresponding corner vertex nodes. the surface which takes the normal at point x as an argument. The surface normal may be calculated as 1 √ (1 + p(x) 2 + q(x) 2 ) (-p(x), -q(x), 1) (2) where p = ∂u/∂x1 (3) and q = ∂u/∂x2 (4) where u is the height of the surface. If there is a unique light source at infinity, and shining in direction - → w =(w1,w2,w3), the pixel intensity is the inner product R( - → n (x)) = w · - → n (x) (5) Hereafter (until section 6), without loss of generality (but as- suming all surface points are visible to both camera and light source) we assume the light source is in the same direction as the camera, which produces an orthogonal projection. 3. MRF formulation to solve SFS This algorithm calculates an optimal set of labels for the height at each corner vertex on the image. A corner vertex occurs at the corner of a pixel; at the intersection of four pixels, one corner vertex represents the height of the surface at that location. Each triplet of vertices describes a unique plane, and the orientation of that plane relative to the direction of the light source allows