Iterated dynamic programming and quadtree subregioning for fast stereo matching Carlos Leung a, * ,1,2 , Ben Appleton a,1,3 , Changming Sun b a Electromagnetics and Imaging, ITEE, The University of Queensland, Brisbane, Qld 4072, Australia b CSIRO Mathematical and Information Sciences, Locked Bag 17, North Ryde, NSW 1670, Australia Received 18 February 2005; received in revised form 12 November 2007; accepted 23 November 2007 Abstract The application of energy minimisation methods for stereo matching has been demonstrated to produce high quality disparity maps. However, the majority of these methods are known to be computationally expensive requiring minutes of computation. In this paper, we propose a fast minimisation scheme that produces high quality stereo reconstructions for significantly reduced running time, requiring only a few seconds of computation. The minimisation scheme is carried out using our iterated dynamic programming algorithm, which iterates over entire rows and columns for fast stereo matching. A quadtree subregioning process is also used for efficient computation of a matching cost volume where iterated dynamic programming operates on. Ó 2008 Published by Elsevier B.V. Keywords: Stereo matching; Energy minimisation; Iterated dynamic programming; Quadtree subregioning 1. Introduction The study of computational stereo has undergone inten- sive research since its inception in the 1970s. Stereo match- ing is the main step for the recovery of the 3D structure of the scene given a pair of images. By matching primitives such as points, curves and regions between the stereo pair, such that the matched primitives are projections of the same 3D identity in the scene, a disparity map of the scene can be computed. While simple local correspondence methods have the advantage of low computational complexity they suffer from high sensitivity to matching ambiguity and the choice of matching metric. Reconstructions based on simple local correspondence can be improved by incorporating global constraints and structural information into the stereo matching process. One such class of global correspondence methods are those based on dynamic programming (DP). Since the dynamic programming framework allows efficient optimal solutions, it has been applied to locate the path of minimum matching cost for each scanline of the image [1–3]. However, since DP is typically applied independently to each scanline, methods that employ this technique suffer from interscanline inconsistencies. Several studies have addressed this issue by applying postprocessing to itera- tively improve the reconstruction, enforcing interscanline constraints. These techniques include minimising the num- ber of horizontal and vertical discontinuities [4], estimating vertical slopes [5], and using edge maps [6]. These methods attempt to retain the computational benefits of a dynamic programming formulation while avoiding the problem of horizontal streaking. However, while these heuristics improve interscanline consistencies they do not entirely solve the problem. 0262-8856/$ - see front matter Ó 2008 Published by Elsevier B.V. doi:10.1016/j.imavis.2007.11.013 * Corresponding author. Tel.: +61 7 3836 1606. E-mail addresses: carlos.leung@gmail.com (C. Leung), appleton@ google.com (B. Appleton), changming.sun@csiro.au (C. Sun). 1 Carlos Leung and Ben Appleton are supported by the Australian Postgraduate Award and CSIRO Mathematical and Information Sciences. 2 Present address: Suncorp, P.O. Box 1453, Brisbane, Qld 4001, Australia. 3 Present address: Google Inc., 201 Sussex Street, Sydney, NSW 2000, Australia. www.elsevier.com/locate/imavis Available online at www.sciencedirect.com Image and Vision Computing 26 (2008) 1371–1383