Statistical Pose Averaging with Non-isotropic and Incomplete Relative Measurements Roberto Tron and Kostas Daniilidis GRASP Lab, University of Pennsylvania, Philadelphia, PA, USA {tron,kostas}@cis.upenn.edu Abstract. In the last few years there has been a growing interest in optimization methods for averaging pose measurements between a set of cameras or objects (obtained, for instance, using epipolar geometry or pose estimation). Alas, existing approaches do not take into considera- tion that measurements might have different uncertainties (i.e., the noise might not be isotropically distributed), or that they might be incomplete (e.g., they might be known only up to a rotation around a fixed axis). We propose a Riemannian optimization framework which addresses these cases by using covariance matrices, and test it on synthetic and real data. Keywords: Pose averaging, Riemannian geometry, Error propagation, Anisotropic filtering, Incomplete measurements. 1 Introduction Consider N reference frames, each representing, e.g., the pose of a camera or of an object. Assume that we can completely or partially measure the relative rigid body transformations for a subset of all possible pairs of frames (see Figure 1). Our goal is to combine all these measurements and obtain an estimate of the position of each frame with respect to some global reference. In order to do so, if there are enough measurements available, we can exploit the geometric constraints induced by combining the poses in cycles. This usually takes the form of an optimization problem that “averages” the poses. However, we need to take into account that the estimates might be partially erroneous or unknown. For instance, the noise in the estimated translations could be higher in some direction, or two rotations could be constrained to be coplanar and have the same z -axis, but differ otherwise. If these errors and ambiguities are not correctly handled, they could propagate and bias the entire result. However, if correctly combined, the different measurements can complement each other into a complete and accurate solution. In this paper we propose to explicitly model non-isotropic noise and incomplete poses through the use of covariance matrices. This is similar to the idea of gradient-weighted least- squares fitting in the statistics literature [22]. More in detail, we propose to proceed as follows: 1. Estimate the relative rigid body transformations between pairs of references and their uncertainties or ambiguities. D. Fleet et al. (Eds.): ECCV 2014, Part V, LNCS 8693, pp. 804–819, 2014. c  Springer International Publishing Switzerland 2014