BMVC 2011 http://dx.doi.org/10.5244/C.25.95 JEN et al.: ADAPTIVE SCALE SELECTION FOR HIERARCHICAL STEREO 1 Adaptive Scale Selection for Hierarchical Stereo Yi-Hung Jen dedanann@cs.unc.edu Enrique Dunn dunn@cs.unc.edu Pierre Fite-Georgel pierre.georgel@gmail.com Jan-Michael Frahm jmf@cs.unc.edu 3D Computer Vision Group Computer Science Department University of North Carolina Chapel Hill, NC, USA Abstract Hierarchical stereo provides an efficient coarse-to-fine mechanism for disparity map estimation. However, common drawbacks of such an approach include the loss of high frequency structures not observable at coarse scale levels, as well as the unrecover- able propagation of erroneous disparity estimates through the scale space. This paper presents an adaptive scale selection mechanism to determine a suitable resolution level from which to begin the hierarchical depth estimation process for each pixel. The pro- posed scale selection mechanism allows us to robustly implement variable cost aggre- gation in order to reduce the variability of the photo-consistency measure across scale space. We also incorporate a weighted shiftable window mechanism to enable error cor- rection during coarse-to-fine depth refinement. Experiments illustrate the effectiveness of our approach in terms of disparity accuracy, while attaining a computational efficiency compromise between full resolution and hierarchical disparity map estimation. 1 Introduction Stereo disparity map estimation entails determining the set of image-wide pixel-level cor- respondences among a pair of input images. Canonical hierarchical stereo approaches se- quentially process different levels of the input imagery scale space representation, using previously computed disparity estimates to constrain the search range at the current level. Accordingly, hierarchical stereo offers a computationally efficient framework for coarse- to-fine depthmap estimation and refinement. However, by virtue of the sampling theorem, disparity estimates for fine-grain scene structures (i.e. originally imaged at spatial frequen- cies greater than half the sampling frequency of the coarsest level of the scale space) may not be properly estimated. Also, the image smoothing and sub-sampling inherent to dis- crete scale space generation may induce erroneous disparity estimates propagated along the scale space during the coarse-to-fine refinement process. Accordingly, the two main factors hindering the correctness of hierarchical stereo approaches are 1) the variability at different c  2011. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.