Generalized Subgraph Preconditioners for Large-Scale Bundle Adjustment Yong-Dian Jian, Doru C. Balcan, and Frank Dellaert College of Computing, Georgia Institute of Technology {ydjian,dbalcan,dellaert}@cc.gatech.edu Abstract. We propose the Generalized Subgraph Preconditioners (GSP) to solve large-scale bundle adjustment problems efficiently. In contrast with previous work using either direct or iterative methods alone, GSP combines their advantages and is significantly faster on large datasets. Similar to [12], the main idea is to identify a sub-problem (subgraph) that can be solved efficiently by direct methods and use its solution to build a preconditioner for the conjugate gradient method. The dif- ference is that GSP is more general and leads to more effective preconditioners. When applied to the “bal” datasets [2], our method shows promising results. 1 Introduction Large-scale visual modeling with Structure from Motion (SfM) algorithms is an impor- tant problem. Recently, systems capable of handling millions of images have been built to realize this task [1,13,23], enabling automated 3D model generation from unstruc- tured internet photo collections. Bundle adjustment is used to find the optimal estimates of camera poses and 3-D points [26]. Mathematically speaking, it refers to the problem of minimizing the total reprojection error of the 3-D points in the images. The classical strategy to solve this problem is to apply a damped Newton’s method (e.g., Levenberg-Marquardt) and solve the reduced camera system by Cholesky factorization. However, this strategy does not scale well because the memory requirement of factorization methods grows quadrati- cally with the number of variables in the worst case. Several recent works suggest using iterative methods such as the conjugate gradient (CG) method to solve the linear systems arising in bundle adjustment, as its memory requirement grows only linearly with the number of variables. The convergence speed of the CG method depends on how well conditioned the original problem is [21]. Hence having a good preconditioner is crucial to make CG converge faster, yet most of the pre- vious approaches [2,7,8,14] apply only standard preconditioning techniques, neglecting to exploit SfM-specific constraints. In robotics, Dellaert et al. [12] proposed the Subgraph-Preconditioned Conjugate Gradients method (SPCG), which aims to combine the advantages of direct and itera- tive methods to solve 2-D Simultaneous Localization and Mapping (SLAM) problems. The main idea is to pick a subset of measurements that can be solved efficiently by direct methods, and use it to build a preconditioner for the CG method. They show that SPCG is superior to using either direct or iterative methods alone. However, for the