3D Surface Reconstruction Using Graph Cuts with Surface Constraints ⋆ Son Tran and Larry Davis Dept. of Computer Science, University of Maryland, College Park, MD 20742, USA {sontran, lsd}@cs.umd.edu Abstract. We describe a graph cut algorithm to recover the 3D ob- ject surface using both silhouette and foreground color information. The graph cut algorithm is used for optimization on a color consistency field. Constraints are added to improve its performance. These constraints are a set of predetermined locations that the true surface of the object is likely to pass through. They are used to preserve protrusions and to pursue concavities respectively in the first and the second phase of the algorithm. We also introduce a method for dealing with silhouette uncer- tainties arising from background subtraction on real data. We test the approach on synthetic data with different numbers of views (8, 16, 32, 64) and on a real image set containing 30 views of a toy squirrel. 1 Introduction We consider the problem of reconstructing the 3D surface of an object from a set of images taken from calibrated viewpoints. The information exploited includes the object’s silhouettes and its foreground color or texture. 3D shape recovery using silhouettes constitutes a major line of research in computer vision, the shape-from-silhouette approach. In methods employing silhouettes only (see e.g. [1]), voxels in a volume are carved away until their projected images are consistent with the set of silhouettes. The resulting object is the visual hull. In general, the visual hull can be represented in other forms such as bounding edges ([2]), and can be reconstructed in a number of different ways. The main drawback of visual hulls is that they are unable to capture concavities on the object surface ([3]). A 3D surface can also be reconstructed using color or texture consistency between different views. Stereo techniques find the best pixel matching between pairs of views and construct disparity maps which represent (partial) shapes. Combining from multiple stereo maps has been studied, but is quite complicated ([4]). Space carving ([5]) and recent surface evolution methods (e.g. [6], [7]) use a more general consistency check among multiple views. The combination of both silhouettes and foreground color to reconstruct an object’s surface has been studied in a number of recent papers ([7], [8], [9]). ⋆ This work is supported by the NSF grant IIS-0325715 entitled ITR: New Technology for the Capture, Analysis and Visualization of Human Movement. A. Leonardis, H. Bischof, and A. Prinz (Eds.): ECCV 2006, Part II, LNCS 3952, pp. 219–231, 2006. c  Springer-Verlag Berlin Heidelberg 2006