MR and DP Based Specular Surface Reconstruction Ravindra Redddy K Center for Visual Inforation Technology Interational Institute of Inforation Technology Hyderabad, India - 500032 Email: rrk.ravindra@gmail.com Abstract-This paper addresses the problem of reconstruction of specular surfaces using a combination of Dynamic Program ming and Markov Random Fields formulation. Unlike traditional methods that require the exact position of environment points to be known, our method requires only the relative position of the environment points to be known for computing approximate normals and infer shape from them. We present an approach which estimates the depth from dynamic programming routine and MRF stereo matching and use MRF optimization to fuse the results to get the robust estimate of shape. We used smooth color gradient image as our environment texture so that shape can be recovered using j ust a single shot. We evaluate our method using synthetic experiments on 3D models like Stanford bunny and show the real experiment results on golden statue and silver coated statue. I. INTRODUCTION Reconstruction of the specular (mirror-like) surfaces is a challenging problem and drawn considerable attention in recent years. As, the observed images of a specular object is a fnction of the object shape as well as the exact nature of environment that surounds the object. Traditional methods of specular object reconstruction follows one of the two approaches: i) Exert complete control over the environment (coded environments) and recover the shape of the object with very less restrictions on the object shape except self refections, and ii) Assume that the environment is highly unconstrained and use assumed object properties (integrability and smoothness) to disambiguate possible hypotheses that arise fom the reconstruction. We explore a method that assumes far lesser control of the environment while allowing arbitrary object shapes (except self refections). In this paper, we attempt to reconstruct the surface using optimization famework on matching two stereo images. This method mainly comprises of three steps, Firstly, the images which are captured in a artifcial setup with very minimal constraints on the perfect calibration of the environment and camera capturing mode and given as input for pre-processing. The pre-processing stage involves, extraction of approximate normals at each imaged object points. Secondly we apply two schemes for extracting depth fom stereo, one being the dy namic programming approach and the second by formulating the problem into optimization problem and use Loopy Belief Propagation (LBP). Finally, a simple LBP approach to fse the results of two algorithms to get the optimized result. We evaluate our method using synthetic experiments of the rigid objects and show our results of reconstruction of ganesha statue captured in a controlled environment. Anoop Namboodiri Center for Visual Inforation Technology Interational Institute of Information Technology Hyderabad, India - 500032 Email: anoop@iiit.ac.in This paper contains two contributions. First we develop an experimental setup which can be used to extract object features and thus facilitating us adapt the methods of lambertian surface reconstruction for specular object reconstruction. Secondly, we propose an M based integration famework which takes the depth estimates fom various methods and gives the robust estimate of depth of the object. The remainder of the paper was organized as follows, literature review is discussed in Sec. II. In Sec. III we present our reconstruction algorithm. Sec. IV contains discussion on our method and fnally, we present our results in Sec. V for both synthetic and real objects. II. RELATED WORK Over the past few years much progress has been made towards solving the specular object reconstruction problem. Early works include study of specularities [1]-[3] and use of calibrated patters [4], [5] to estimate the normals. Tirani et al. [5], integrated the norals around a seed point and using a global self-coherence measure to estimate the corect depth for the seed point. Geometrical methods like multi-view, stereopsis and single view techniques gives dense reconstruction but it requires precise knowledge of environment points. In [6], Nehab et al. used a calibrated setup to fnd the specularities and in tur normals at each point They used a dynamic programming for matching sequences. In [7], using light-path triangulation Kutulakos et al. showed that it is practically impossible to reconstruct the surface if the object self refects more than once (refects more than twice). Higher order diferential geometry of the surface is ex plored by [8], [9]. Adato et al. [lO] exploited the specular fow and forulated the reconstruction into linear PDEs. Aswin C. et al. [11], [12] proposed a method of fnding image invariants for smooth specular objects and in tur sparse refection corespondences. With a fnite motion of object, fxed camera and uncalibrated environment. Estimating stereo corespondences using Markov Random Fields (MRF) optimization for fnding disparities has been extensively used for lambertian surfaces. Most moder ap proaches fame the problem as inference on a Markov random feld and utilize global optimization techniques to estimate the depth/ disparity at each object pixel in the image. Various opti mization techniques like, Iterative Conditional Methods, graph cuts, loopy belief propagation using sum-product and max product were tried in [13]-[16] and compared [14] by various