Non-linear spatial warping of endoscopic images: an architectural perspective for real time applications Vijayan K. Asari Department of Electrical and Computer Engineering, Old Dominion University, 231 Kaufman Hall, Norfolk, VA 23529, USA Received 14 August 2001; revised 7 January 2002; accepted 1 February 2002 Abstract Images captured with a typical endoscope show spatial distortion, which necessitates spatial warping for subsequent analysis. In this paper, an ef®cient architecture for an embedded system for the real-time correction of barrel distortion in endoscopic images is proposed. The spatial warping procedure follows a methodology based on least-squares estimation to correct the non-linear distortion in the endoscopic images. A mathematical model of polynomial mapping is used to map the images from distorted image space onto the warped image space. The model parameters include the expansion polynomial coef®cients, distortion centre, and corrected centre. The spatial warping model is applied to several gastrointestinal images. The spatial warping algorithm is mapped onto a linear array of simple processing elements with each element of a particular segment communicating with its nearest neighbours. Currently, a prototype of the VLSI architecture for an image of size 256 £ 192 is being designed and built. The functional simulation results obtained in the warping architecture are encouraging. The VLSI based system will facilitate the use of a dedicated module that could be mounted along with the endoscopy unit. q 2002 Elsevier Science B.V. All rights reserved. Keywords: Endoscopic images; Spatial warping; Polynomial mapping 1. Introduction Electronic video-endoscopy has become one of the commonly accepted forms of diagnostic and therapeutic procedures due to the advent of miniature CCD cameras and associated microelectronics. Video-endoscopes facili- tate observation, documentation, and electrical manipula- tion of the images of internal structure of the gastrointestinal tract. In these endoscopes, cameras with wide viewing angle lens ®sh-eye lens) are used to enhance the imaging capability, which permits capturing of larger ®eld in a single image [1]. However, it has been noted that the images obtained from electronic endoscopes show barrel type spatial distortion due to wide-angle con®guration of the camera lens. Barrel distortion introduces non-linear changes in the image due to which image areas near the distortion centre are compressed less while areas farther from the centre are compressed more. Because of this, the outer areas of the image look signi®cantly smaller than their actual size. This inhomogeneous image compression intro- duces signi®cant errors in the results obtained during feature extraction. Continuous estimation of quantitative para- meters such as area and perimeter is of considerable importance while performing clinical endoscopy. Unless the distortion is corrected, estimation errors could be very large [2,3]. In addition, the distortion causes complications while using token matching techniques for pattern recogni- tion. Spatial warping is also a prerequisite for the camera calibration to obtain extrinsic and intrinsic camera para- meters [4,5]. Several researchers have presented various mathematical models of the image distortion and techniques to ®nd the model parameters to complete the spatial warping proce- dure. Tsai [6] proposed a radial lens distortion model that describes a two-dimensional image correction technique. A prism distortion model was used in Ref. [7] to correct the tangential distortion in an image. Nomura et al. [8] presented a calibration technique for high distortion TV camera lens. But this method requires a precise placement of the calibration chart. Thus, a small shift of the chart prompts considerable errors in spatial warping. Weng et al. [9] has explained radial, decentring and thin prism type of distortions and techniques to model them mathemati- cally. All the earlier models give reasonable results for images obtained from cameras with normal viewing objec- tive lens but these models are not effective for electronic endoscopes which use wide-angle lens camera. Smith et al. [10] gave a formulation in which distortion was assumed to Microprocessors and Microsystems 26 2002) 161±171 0141-9331/02/$ - see front matter q 2002 Elsevier Science B.V. All rights reserved. PII: S0141-933102)00010-8 www.elsevier.com/locate/micpro E-mail address: vasari@odu.edu V.K. Asari).