MVA2002 lAPR Workshop on Machine Vision Applications, Dec. 11 - 13,2002, Nara- ken New Public Hall, Nara, Japan 3-3 1 Shape from Silhouette and Neural Network Based Optimization Haruki Kawanaka* Yuji Iwahorit Graduate School of Engineering, Center for Information and Media Studies Nagoya Institute of Technology Nagoya Institute of Technology Akira Iwatat Faculty of Engineering Nagoya Institute of Technology Abstract In this paper, a new approach is proposed to re- cover the shape for the restricted observation with the limited rotation angle. This is achieved by combining Shape-from-silhouette and the Hopfield neural network based optimization technique. Under the condition that the number of the observed images is restricted with the limited rotation angle, the original Shape-from-silhouette gives poor result, while the HF-NN optimization gives the high performance with the exact shape through the formulation of the partial derivatives of height and gradient. Further, the approach is quite empirical in that no explicit assumptions are used for the specific surface reflectance function. RBF neural network is used to estimate the image irradiance (i.e. reflectance map R) in the optimization process. Then, computer simulation evaluates the accuracy of our method. Moreover, the experiment by the real object is shown and the effectiveness of the proposed method is demonstrated. 1 Introduction In computer vision, it is an important problem to obtain the 3D-shape from the observed images. Shape- from-silhouette [I] is a method to recover the shape from many images t,aken at many view points for an object. In general, Shape-from-silhouette needs the ob- servation from 360 degrees rotation with the small ro- tation angle to get the detailed shape. However it has the problem to catch the local concave shape. When the movement of a viewpoint is restrictred or the fine observation is difficult, the resulting error becomes very large. In this paper, a new approach is proposed to re- cover the shape for the restricted observation with the limited rotation angle. This is achieved by com- bining Shape-from-silhouette and the neural network based optimization technique. Here, Hopfield like Neu- ral Network [2] (HF-NN) is used for the optimization. HF-NN uses the convex full of the object obtained from Shape-from-silhouette as the initial state. Then, it op- timizes the energy function. In this paper, we formu- 'Address: Gokiso-cho, Showa-ku, Nagoya, 466-8555, Japan. Email: harukimcenter .nitech. ac. jp t~mail: ivahoriacenter .nitech.ac. jp $Email: ivata0elcom.nitech.a~. jp Figure 1: Observation System late the energy function for the shape recovery to get the exact shape using the shading information. The proposed approach takes the advantages that it can catch the detailed shape from the limited number of the observed images and it is not necessary to take the observation from the whole rotation. The method obtains the surface data not the volume data. HF-NN also has an advantage to save the calculation time with a parallel comput,er. Further, the approach is quite empirical in that no explicit assumptions are used for the specific surface reflectance function. Radial basis function neural network (RBF-NN) is used to estimate the image irradiance in the optimization process. 2 Obtaining 30 shape 2.1 Convex Full from Silhouette Fig.1 shows the observation system. A camera and a light source are fixed, instead, an object is put on a turntable. Parallel light source is illuminated from the viewing direction and the orthographic projection is assumed. Shape-from-silhouette tries to get the shape using the inverse projection from the silhouette of each im- age, and the resulting shape becomes strictly a convex full of the object under the condition that the range of the rotation is restricted and the rotation step is taken large. This is shown in Fig.2.