Planar Shape Recognition across Multiple Views Sujit Kuthirummal, C. V. Jawahar, P. J. Narayanan Centre for Visual Information Technology International Institute of Information Technology Gachibowli, Hyderabad, India. 500 019. sujit@gdit.,jawahar@,pjn@ iiit.net Abstract Multiview studies in Computer Vision have concentrated on the constraints satisfied by individual primitives such as points and lines. Not much attention has been paid to the properties of a collection of primitives in multiple views, which could be studied in the spatial domain or in an ap- propriate transform domain. We derive an algebraic con- straint for planar shape recognition across multiple views based on the rank of a matrix of Fourier domain descriptor coefficients of the shape in different views. We also show how correspondence between points on the boundary can be computed for matching shapes using the phase of a mea- sure for recognition. 1 Introduction Multiview analysis of scenes is an active area in Com- puter Vision today. The structure of points and lines as seen in two views attracted the attention of computer vision re- searchers in the eighties and early nineties [5, 1, 3]. Simi- lar studies on the underlying constraints in three views fol- lowed [6, 2]. The structure of greater than four views has also been studied [7, 8]. Two excellent textbooks have re- cently appeared focusing on multiview geometry for Com- puter Vision[1, 3]. The mathematical structure underlying multiple views has been studied with respect to projective, affine, and Euclidean frameworks of the world with amaz- ing results. Multiview studies have focussed on how geometric prim- itives such as points, lines and planes are related across views. Specifically, the algebraic constraints satisfied by the projection of such primitives in different views have been the focus of intense studies. The multilinear relation- ships that were discovered have been found to be useful for a number of tasks, such as view transfer, geometric recon- struction and self calibration. The richness of the informa- tion present among the geometric primitives in a collection of them has not attracted a lot of attention. Such proper- ties are difficult to capture in the spatial domain but can be extracted with relative ease in a transform domain. The analysis of boundary shapes in multiple views using Fourier domain descriptors can provide structure not explicit in the geometric space and provide interesting handles for solving problems like object recognition and view transfer. The properties of collections of primitives in multiple views are studied in this paper. Specifically, we look at the situation of viewing a planar shape from different view- points. Recognizing objects from diverse viewpoints is es- sential to interpreting the structure and meaning of a scene. We use a Fourier domain representation for the boundary of the object and derive recognition constraints the projec- tions of the object must satisfy in multiple views. These constraints are in the form of the rank of the matrix of the descriptor coefficient values. We present the basic problem formulation in the next section. Numerical results to validate the theoretical claims are presented in Section 3, along with some discussions on the underlying issues. Section 4 presents a few concluding remarks. 2 Problem Formulation We are interested in exploiting the relationships between points on the shape boundary in the domain of a Fourier descriptor. Affine homographies have been studied in the Fourier domain [4], in which the boundary points were rep- resented as complex numbers. We need a richer representa- tion to linearize the affine homography relation and so use a vector of complex numbers as our descriptor for points on the boundary of a shape. Let be the homogeneous coordi- nates of points on the closed boundary of a planar shape. The shape is represented by a sequence of vectors of com-