Constrained Segmentation of Complex Models for Image-Based Texture Editing Ioana Boier-Martin IBM T. J. Watson Research Center ioana@us.ibm.com Holly Rushmeier Yale University holly@acm.org Richard Giantisco Freelance 3D Artist ognhisclub@yahoo.com Abstract We present a constrained segmentation method for the partitioning of complex textured 3D models. Our method automatically iden- tifies large regions of reduced metric distortion suitable for texture mapping. The boundaries of the regions conform to salient features of the model and respect user-imposed constraints. We illustrate the application of this method to virtual paint restoration through image-based texture editing. Keywords: 3D segmentation, texture editing, virtual restoration 1 Introduction We focus on the digital restoration of surface paint on objects through editing of the texture maps associated with them. We present a method for processing scanned data so as to simplify sub- sequent texture map editing. With our method the editing can be performed in a standard 2D image editor, circumventing the poten- tial distortion introduced by 3D painting. Our method greatly re- duces the resource requirements for editing, as well as the training time needed to master the software tools used for restoration. 2 Model Segmentation A common 3D scanning pipeline produces a geometric object and a set of images that can be processed into illumination-independent overlapping views [Bernardini and Rushmeier 2002]. The overlap- ping images are combined into a single non-redundant texture map and texture coordinates are computed for the geometry. A typical procedure to compute a mapping from the 3D model to the 2D tex- ture consists of the following steps: (a) model segmentation into regions that can be flattened; (b) region texturing by projecting the corresponding captured and processed views onto the geometry of each region; and (c) packing of the region textures into a single texture map and padding them to avoid seams. We address the segmentation problem. Existing methods either store the texture triangle-by-triangle or distort it to impose absolute metric uniformity. Consequently, the texture maps cannot be edited directly as model features are either broken up or unnaturally dis- torted. Instead, the object must be imported into a 3D object editing system. To accommodate interactive painting rates the detailed ge- ometry must be simplified leading to severely distorted textures. We propose a segmentation method that avoids such excessive dis- tortion. It automatically extracts large height-field regions with boundaries conforming to salient model features and user-imposed constraints. It differs from previous work which either requires the user to manually specify an entire partition or runs fully automati- cally without regard for user-imposed constraints. We make novel use of K-means clustering [MacQueeen 1967] to generate almost flat regions conforming to features (see Fig. 1). The main steps of our method are: 1. Compute face normals across the model. 2. Smooth the normal vector field. 3. Perform a K-means quantization in normal space taking into account the constraints. Figure 1: Constrained segmentation followed by image-based editing (edited re- gions are shown numbered before and after editing; editing results are marked ”E”). 3 Results Fig. 1 shows the result of segmenting a scanned Egyptian stat- uette (data courtesy of CultNat – www.cultnat.org). The model has 112, 478 faces and was segmented into 30 regions (top right) in 3.6 minutes. User-imposed constraints (red curves; top left) ensure that the face, chest, and hair of the model are properly separated to allow subsequent editing (i.e., without constraints the semantics of these parts is lost and they are arbitrarily split between regions, making it difficult to edit them). The middle row illustrates the corresponding texture map (note clearly identifiable head and chest textures). A comparison of the model before and after paint restoration is shown at the bottom (see also the accompanying video clip). References BERNARDINI , F., AND RUSHMEIER, H. 2002. The 3D model acquisition pipeline. Computer Graphics Forum 21, 2, 149–172. MACQUEEEN, J. 1967. Some methods for classification and analysis of multivariate observations. In Proc. 5th Berkeley Symp. Math Statistics and Prob., 281–297.