Implementation of a robust 3D-image reconstruction algorithm on a TMS320C67x DSP Marc Leeman, Rudy Lauwereins, Marc Pollefeys ACCA / ESAT K.U.Leuven Kardinaal Mercierlaan 94 3001 Leuven, Belgium firstname.lastname@esat.kuleuven.ac.be Fax: +32 16 32 19 69 Introduction 3D-image reconstruction is both an interesting as well as a challenging topic to work on. On one hand, there is an ever increasing interest in the use of these techniques in different kinds of application fields. On the other hand, most of the available techniques are limited by their extensive and as such expensive setup or the ability to capture and reconstruct only small artifacts. The algorithm developed by Pollefeys et al.[1] solved these issues by starting from 2D images made with a standard hand-held camera. Due to the heavy computational requirements of the algorithm, the most powerful floating point DSP available was chosen. When the project started, this was the Texas Instruments TMS329C67x, which was soon to become in production. This paper will focus on the implementation of one of the first steps in the 3D-image reconstruction algorithm. Transformations Problem Description When implementing the algorithm as is on a DSP, a number of issues are critical. First of all, the data requirements are quite large and the physical memory hierarchy on the DSP board is a limiting factor. Another issue is that for fast access to the data, the values have to be directly accessible to the DSP, with a minimum on overhead. For this reason, the appropriate data has to be in internal memory. The ’C6701 only has 64 kB on-chip memory. A strategy has to be found to get the values to the DSP at the right time, halting it as little as possible. Finally, accessing external memory is very power consuming and should be limited as much as possible. This effect is very important for devices with an independent power supply (battery). In the ideal case, the data is transferred only once to internal memory and then processed. During the implementation, a number of transformations were immediately identified as critical areas in memory usage for the target platform. In what follows, the transformations of the HARRIS corner detector for finding significant points in the pictures are elaborated. The HARRIS corner detector When reconstructing an 3D scene, not the entire initial images are of interest, but only those features that can be matched between different pictures. Only then they provide 3D information. From the existing interest point detectors, the HARRIS corner detector gives the best results according to the two following relevant criteria [3]: • the extraction of the points should be as much as possible independent of camera pose and illumination changes. • the neighbourhood of the selected points should contain as much information as possible to allow matching. Finding the corners with the HARRIS corner-detector is done in different sub-steps. To retain the best corners, all pixels in the images are given their cornerness value. This property is an indication for a certain point of how much it satisfies the cornerness criteria (a strong gradient in both directions). The cornerness value is defined as: 2 2 ) ( 04 . 0 ) ( yy xx xy yy xx f f f f f c + × - - × = with c the cornerness factor; f(x,y) the intensity of a pixel at coordinates (x,y). f xx , f yy and f xy are the second derivatives to x, y and to x and y. In practice, finite derivatives are computed, using 1D Laplacian masks with a size of 5. After the detection and selection of the most prominent corners, Harris’ algorithm adds one more step for finding the sub-pixel precision. A point has an intensity f(x,y). The intensity value is only defined for x and y values that are at pixel coordinates. When four neighbouring pixels are found with high cornerness values f(x,y) (the most prominent corners are found for the pixels with the highest cornerness values), their function values can be used to find a maximum with quadratic fitting somewhere in between these four pixels. This maximum can be considered as a corner with sub-pixel precision. Only images without texture or very textured images can cause problems. In the first case, not enough corners are found, in the latter too many unreliable corners are found. To ensure that corners are retained over the entire picture, the image is split in areas and in each area a portion of the desired number of corners is searched. Proposed Memory Optimisations In the following paragraphs, different approaches and their effect on the used memory will be discussed. This starts from the original implementation to a strategy that produces a minimum on intermediate storage. The disadvantages are explained and will lead to another solution that optimally uses the available resources while compromising the execution time as little as possible. But before actually going deeper into this, a word on the used data structures is due.