How to Put Probabilities on Homographies Evgeni Begelfor and Michael Werman Abstract—We present a family of “normal” distributions over a matrix group together with a simple method for estimating its parameters. In particular, the mean of a set of elements can be calculated. The approach is applied to planar projective homographies, showing that using priors defined in this way improves object recognition. Index Terms—Homography, lie groups, normal distribution, Bayesian statistics, geodesics. æ 1 INTRODUCTION AND OUTLINE ONE reason to describe a distribution of homographies is to introduce a meaningful prior for Bayesian image recognition. Suppose we want to distinguish between N planar objects. The input we are given is a picture of an object imaged from an unknown direction. Thus, the image, D, should be obtained from the original by some homography . Let M be the model, in other words, one of the objects. From Bayes rule we get: P ðM;jDÞ¼ P ðDjM;ÞP ðM;Þ P ðDÞ ¼ ¼ P ðDjM;ÞP ðjMÞP ðMÞ P ðDÞ : ð1Þ Thus, to be fully Bayesian, we need to know P ðjMÞ, that is, a distribution on the group of the homographies of a plane. Many papers, such as [13], [5], deal with means on groups, although all of them are for subgroups of the group of Euclidean motions. In [3], [4], a method for putting a distribution on a Lie group is described. The method is appropriate, though, only when the group is compact (such as SOð3Þ, the group of rotations of the 3D space) or abelian (such as IR n , the group of translations) or direct products of such groups. However, some models involve groups that cannot be represented in such a way; in particular, the views of a planar object from different directions. These are modeled with the group of plane homographies, that is, 3-by-3 matrices where matrices differing only by scalar multiplication represent the same homography. To get rid of this ambiguity, we normalize the matrices to have determinant 1. This group is usually denoted by SL 3 ðIRÞ. Distributions on such groups were treated in [11], but no method for estimating expectation and the parameters of the distribution was described and the absence of the covariance greatly diminishes the ability of the distribution to fit data. In this paper we propose a parametric distribution on such groups, together with simple methods for finding the parameters. The main idea needed in order to define the distribution is as in [3] for the orthogonal group, the geodesic distance on the group. The geodesic distance is used in order to define a mapping from the group G to a linear space where we estimate the parameters of the normal distribution. Another way to look at it is to say that we define an invariant distribution on the group and learn its parameters. The next section shows the need to define the probability using the group structure. In Section 3, the mathematical background needed is described. Section 4 describes the actual algorithms for estimating the parameters of the distribution fitted to given data. The paper finishes with a demonstration of the methods applied to object recognition. 2 MOTIVATION Intuitively, a “normal” distribution on G should have a mean value  and a covariance matrix Æ. In the case of the usual normal distributions on IR n , we know that if a random variable X is translated by t, then the probability translates X  Nð; ÆÞ ¼) ðX þ tÞ Nð þ t; ÆÞ: We would like a similar property to hold for our distributions on G. For example, suppose that we have a planar object and a distribution of camera positions above it (Fig. 1). If the distribution of the homographies from a set of images I to Image 1 is Nð; ÆÞ, the distributions of the homographies from the set of images I to Image 2 is g  Nð; ÆÞ, where g is the homography between Image 1 and Image 2. We would like the parameters to be invariant to the group action, that is, h  Nð; ÆÞ ¼) ðghÞ Nðg  ; ÆÞ: One might try to define a distribution on the group G, for example, SL 3 ðIRÞ, by treating it as a subset of IR 9 . There are a few problems with this approach. First of all, SL 3 ðIRÞ is an 8-dimensional manifold and not 9-dimensional. One might take only eight coefficients of the 3  3 matrix and define the distribution using those. But, in this way, the invariance property doesn’t hold. The solution is to define the distribution using intrinsic features of the group G. We define the distribution with a given mean  by mapping a normal distribution on the tangent space at  to the group itself, while keeping the invariance properties. In Fig. 2, the advantage of our distribution is demonstrated. A set of homographies between a planar object and its image when the camera is randomly placed on a sphere above the plane. The dashed line is the density of one of the coefficients of the usual matrix representation of a homography (as a 3-by-3 matrix normalized to have determinant 1) and the solid line is the density of the corresponding coefficient after our transformation was applied. It can be seen that our distribution is more “normal” and informative. The distribution of the coefficients after transformation is much closer to normal, as is demonstrated by the results of Kolmogorov- Smirnov [7] tests of normality of the coefficients shown in Fig. 2 (smaller numbers imply more normality, the numbers were scaled). 3 MATHEMATICAL BACKGROUND The tools used here come primarily from Lie theory and differential geometry. For more information on these subjects, the reader is referred to [14] or [8]. A Lie group G is a group which is also a smooth manifold, such that multiplication and inversion are smooth. For any point x on a smooth manifold, one has the tangent space to the manifold at x, denoted by T x . Many of the examples of Lie groups are matrix groups, for example, G ¼ SL n ðIRÞ, the set of all n-by-n matrices with real entries and determinant 1. This set has a manifold structure inherited from the natural manifold structure of IR n 2 , the set of all matrices. Every matrix in G has an inverse in G and as the determinant is multiplicative, the product of two matrices in G is in G. Thus, SL n ðIRÞ is a Lie group. As with any smooth manifold and any point on it, if we have a Lie group G, one has the tangent space to the identity element e G which we will denote by G. 1 There exists a map, called the exponential map, 1666 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 10, OCTOBER 2005 . The authors are with the Computer Science Department, The Hebrew University of Jerusalem, Israel 91904. E-mail: {aristo, werman}@cs.huji.ac.il. Manuscript received 13 Apr. 2004; revised 24 Feb. 2005; accepted 2 Mar. 2005; published online 11 Aug. 2005. Recommended for acceptance by D. Fleet. For information on obtaining reprints of this article, please send e-mail to: tpami@computer.org, and reference IEEECS Log Number TPAMI-0175-0404. 1. The reason we pick a special name, G, for the tangent space, which we could denote by T e , is that this space, called the Lie algebra of G, plays an important role in Lie theory, and proofs of some of the following claims use Lie algebra. We chose to omit the definition of Lie algebras and their multiplication in order to keep this exposition as simple as possible. 0162-8828/05/$20.00 ß 2005 IEEE Published by the IEEE Computer Society