Cell-Based Approach for 3D Reconstruction from Incomplete Silhouettes Maarten Slembrouck (B ) , Peter Veelaert, David Van Hamme, Dimitri Van Cauwelaert, and Wilfried Philips imec - TELIN-IPI, Ghent University, St-Pietersnieuwstraat 41, 9000 Gent, Belgium maarten.slembrouck@ugent.be http://telin.ugent.be/~mslembro Abstract. Shape-from-silhouettes is a widely adopted approach to com- pute accurate 3D reconstructions of people or objects in a multi-camera environment. However, such algorithms are traditionally very sensitive to errors in the silhouettes due to imperfect foreground-background esti- mation or occluding objects appearing in front of the object of interest. We propose a novel algorithm that is able to still provide high quality reconstruction from incomplete silhouettes. At the core of the method is the partitioning of reconstruction space in cells, i.e. regions with uniform camera and silhouette coverage properties. A set of rules is proposed to iteratively add cells to the reconstruction based on their potential to explain discrepancies between silhouettes in different cameras. Exper- imental analysis shows significantly improved F1-scores over standard leave-M-out reconstruction techniques. Keywords: Shape-from-silhouettes · 3D reconstruction · Occlusion · Multi-camera 1 Introduction Shape-from-silhouettes algorithms are very sensitive to errors in the provided sil- houettes. Two types of errors are common: inaccurate silhouette boundaries and parts of the silhouette that are missing entirely. Such incomplete silhouettes may be due to errors in the segmentation algorithm, such as foreground/background segmentation, but their primary cause is occlusion. If a static object is positioned between the camera and the moving object, foreground/background segmenta- tion is unable to segment parts of the silhouette. In indoor as well as outdoor environments, occlusion seems to be inevitable. Examples of occluding objects are furniture in an indoor setting or parked cars in outdoor setting. Although in some circumstances, it may be possible to manually mark the occluding objects in camera images, for example, during an interview with fixed cameras, in most applications manual occlusion marking is impractical [2]. Alter- natively, the presence of occluders has also been inferred from depth images, where depth information is provided either by stereo cameras [3, 7–9] or by depth c  Springer International Publishing AG 2017 J. Blanc-Talon et al. (Eds.): ACIVS 2017, LNCS 10617, pp. 530–541, 2017. https://doi.org/10.1007/978-3-319-70353-4_45