Machine Vision and Applications (2009) 20:261–270 DOI 10.1007/s00138-007-0121-z ORIGINAL PAPER A sub-pixel stereo matching algorithm and its applications in fabric imaging Wurong Yu · Bugao Xu Received: 13 February 2006 / Revised: 20 August 2007 / Accepted: 20 November 2007 / Published online: 10 January 2008 © Springer-Verlag 2008 Abstract In this paper, we describe a sub-pixel stereo matching algorithm where disparities are iteratively refined within a regularization framework. We choose normalized cross-correlation as the matching metric, and perform dis- parity refinement based on correlation gradients, which is distinguished from intensity gradient-based methods. We propose a discontinuity-preserving regularization technique which utilizes local coherence in the disparity space image, instead of estimating discontinuities in the intensity images. A concise numerical solution is derived by parameterizing the disparity space with dense bicubic B-splines. Experimen- tal results show that the proposed algorithm performs better than correlation fitting methods without regularization. The algorithm has been implemented for applications in fabric imaging. We have shown its potentials in wrinkle evaluation, drape measurement, and pilling assessment. Keywords Stereo vision · Disparity parameterization · Smoothness constraint · Multiresolution · Fabric imaging 1 Introduction Stereo matching with sub-pixel accuracy is essential to a stereo vision system for realistic surface imaging where fine geometric details need to be recovered. Among a wide variety of matching algorithms (see reviews [5, 6, 20]) developed in last three decades, some of them naturally produce fractio- nal disparity estimates (e.g., gradient-based [14] and level set method [7]), but most methods compute disparities at W. Yu · B. Xu (B ) Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX 78712, USA e-mail: bxu@mail.utexas.edu discrete values (e.g., block matching [3], dynamic program- ming [8] and graph cuts [4]). For the latter case, image upsampling prior to matching [27] or disparity refinement as a post-process [2] is often employed to achieve sub-pixel accuracy. But upsampling would cause computation and memory overloaded, especially when we need to use high- order interpolation functions to reduce aliasing noise in the upsampled image. On the other hand, disparity refinement is more efficient and thus often used in practice. One of the standard refinement methods is fitting a parabola to the neigh- borhood around the peak of correlation function. However, parabola fitting suffers from systematic error called “pixel- locking” effect in which disparity values are pulled toward integers [21]. In addition to systematic error, one usually performs disparity refinement separately at each position of the image, so the refined disparity map is prone to be noisy. To tackle these problems, we propose a sub-pixel matching algorithm which iteratively refines disparity map based on correlation gradients and imposes smoothness constraints on disparity updating. The smoothness constraints are imple- mented within a local coherence-based regularization fra- mework, which is an additional contribution of our work. It should be pointed out that stereo vision is an application- oriented problem, which means it is very hard to develop an algorithm which is general enough to be applicable under any circumstance. The work presented in this paper is also driven by a project on surface imaging of fabrics. Objective evalua- tion of fabric appearance is traditionally based on 2D image analysis (e.g., [33, 34]). However, 2D imaging techniques are sensitive to fabric colors and patterns, and can be affected by the condition of illumination. To overcome these limitations, 3D surface imaging techniques have attracted more atten- tion in last decade. For example, laser profilometer has been used successfully in wrinkle evaluation [1, 22, 35]. This tech- nique is accurate and reliable, but the scanning process is 123