EMBEDDED IMAGE PROCESSING/COMPRESSION FOR HIGH-SPEED CMOS SENSOR R.Mosqueron, J.Dubois and M.Paindavoine Laboratoire Le2i - UMR CNRS 5158, Universite de Bourgogne Aile des Sciences de l’Ingenieur, BP 47870, 21078 Dijon cedex, France phone: + (33) 380393606, fax: + (33) 380395910, email: romuald.mosqueron@u-bourgogne.fr web: www.le2i.com ABSTRACT High-speed video cameras are powerful tools for investigat- ing for instance the biomechanics analysis or the movements of mechanical parts in manufacturing processes. In the past years, the use of CMOS sensors instead of CCDs has made possible the development of high-speed video cameras of- fering digital outputs, readout flexibility and lower manufac- turing costs. In this paper, we proposed a high-speed cam- era based on CMOS sensor with embedded processing. Two types algorithms have been implemented. The compression algorithm represents the first class for our camera and allows to transfer images using serial output link. The second type is dedicated to feature extraction like edge detection, mark- ers extraction, or image analysis, wavelet analysis and object tracking. These image processing algorithms have been im- plemented into a FPGA embedded inside the camera. This FPGA technology allows us to process in real time 500 im- ages per second with a 1, 280H × 1, 024 V resolution. Keywords: CMOS Image Sensor, FPGA, Image Compres- sion, High-speed Video. 1. INTRODUCTION In the past years, the use of CMOS sensors instead of CCDs [1, 2]has made possible the development of industrial high- speed video cameras offering digital outputs, readout flexi- bility and lower manufacturing costs. Two main limitations of these systems may be discussed. First, the huge data flow provided by the sensor cannot be easily transferred or processed and has generally to be stored temporarily in a local fast RAM memory. This RAM is size limited so the recording time in the camera is only a few seconds long. Us- ing an image compression approach, we developed an alter- native solution that allows continuous recording. With this approach, we do not use an embedded RAM memory, and store directly to a connected PC memory. A major advantage is to use the permanent evolution of PC RAM memory. Second, in order to execute in real-time image processing dedicated to tracking or object recognition, the interesting information must be extracted from huge data flow inside of the FPGA. The image processing algorithms have been implemented inside a FPGA and with technology, we have show that it is possible to process in real-time 500 frames per second (resolution 1, 280H × 1, 024V ). In particular, to reach this speed, we implemented on FPGA, fast parallel algorithms dedicated to image compression and image seg- mentation. This paper is organized as follow. Our high-speed camera is described in Section 2. The studied image com- pression and image processing algorithms and their imple- mentations inside the FPGA are introduced in Section 3. Fi- nally conclusions and perspectives are drawn in Section 4. 2. HIGH-SPEED CAMERA DESCRIPTION Nowadays, many image CMOS sensors exist providing a high acquisition flexibility. Moreover, some of them en- able to access simultaneously to several pixels at high fre- quency (up to 66MHz) therefore the input data bandwidth is extremely high. This kind of sensor allows many possibilities in terms of high-speed acquisition and processing. Indeed, in order to process in real-time, simultaneously pixel access is required. For these reasons, we have designed a high-speed camera based CMOS sensor and in order to control this data flow and to process in real-time these informations, a specific unit has been designed based on FPGA component. FPGA features enable to connect a high number of I/O, to design a specific controller adapted to the sensor and finally to achieve real-time processing on large input data flow. Low level image processing are regular, frequently same task can be done simultaneously on several pixels or different regions of the image. The FPGA architecture and hardware ressources are specially adapted to these operations therefore processing time can be significantly reduce and data flow bottleneck re- move. In the literature, almost high-speed cameras are devel- oped with embedded memory (optronis CAMRECORD600 [3], motion Blitz cube Eco2 Mikrotron [4]and others) or with parallel output (camera link) but the host PC need a frame grabber (VDS CMC1300 [5] or Basler A504 [6]). To obtain long sequences, the memory is very large and therefore very expensive. We propose a new solution based on the suppres- sion of the embedded memory and direct data transfer into the external memory on the host PC connected to the camera output. By using PC memory, the camera benefits of evolu- tion in terms of size and frequency. The camera output has to be the most simple as possible with low cost like a standard serial interface like firewire or USB. We used the MT9M413 high-speed CMOS image sensor from Micron in order to design our high-speed camera. The main features of this image sensor are a high frame rate (500 images per second at full size frame (1, 280H × 1, 024 V )), the output with 10-bit digital through 10 parallel ports, a high output data rate at 660Mbs,the shutter exposure time with a minimum of 100ns. In relation to the image data rate and im- age resolution we have selected the VIRTEX-II XC2V3000 FPGA from Xilinx with 14,336 logical blocks (slices), 96 dedicated 18-bit× 18-bit multiplier blocks and 18 Kbit dual- port RAM (BRAM) and 720 I/O pads. USB2.0(Universal Serial Bus version 2.0) is a interesting solution for our digi- tal camera because it provides true plug and play installation and it is an hot-plugging material. Our high-speed camera 14th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, September 4-8, 2006, copyright by EURASIP