Accelerating image recognition on mobile devices using GPGPU Miguel Bordallo L´opez a , Henri Nyk¨anen b , Jari Hannuksela a , Olli Silv´ en a and Markku Vehvil¨ainen c a Machine Vision Group, University of Oulu, Oulu, Finland b Visidon Ltd, Oulu, Finland c Nokia Research Center, Tampere, Finland ABSTRACT The future multi-modal user interfaces of battery-powered mobile devices are expected to require computation- ally costly image analysis techniques. The use of Graphic Processing Units for computing is very well suited for parallel processing and the addition of programmable stages and high precision arithmetic provide for opportu- nities to implement energy-efficient complete algorithms. At the moment the first mobile graphics accelerators with programmable pipelines are available, enabling the GPGPU implementation of several image processing algorithms. In this context, we consider a face tracking approach that uses efficient gray-scale invariant texture features and boosting. The solution is based on the Local Binary Pattern (LBP) features and makes use of the GPU on the pre-processing and feature extraction phase. We have implemented a series of image processing techniques in the shader language of OpenGL ES 2.0, compiled them for a mobile graphics processing unit and performed tests on a mobile application processor platform (OMAP3530). In our contribution, we describe the challenges of designing on a mobile platform, present the performance achieved and provide measurement results for the actual power consumption in comparison to using the CPU (ARM) on the same platform. Keywords: Computer vision, cell phone, image recognition, graphics processing unit, GPGPU 1. INTRODUCTION The future multi-modal user interfaces of battery-powered mobile devices are expected to require energy-efficient image analysis for tasks such as face tracking and gesture recognition. These low level image processing operations are computationally costly and often need to be performed in real time. Currently, mobile devices make use of SIMD-units for operation parallelization. Even though the SIMD computing model presents high flexibility, it relies on a CPU with control code execution, while a Graphics Processing Unit can be treated as an independent entity. This provides for attractive opportunities to implement complete algorithms on the GPU. In addition, the energy efficiency of the GPU could reduce the power needs of image analysis tasks in mobile devices. Figure 1 depicts a typical organization of a mobile platform. The camera is interconnected with the main processor that has access to both the main memory and the graphics memory. Since there is no straight interface that links the camera with the graphics memory, the camera data needs to be transfered from the main CPU to the GPU. Using Graphics Processing Units (GPUs) to perform computationally intensive tasks has become popular in many industrial and scientific applications. As GPU computing is well suited for parallel processing, it is also a very interesting solution for accelerating image recognition solutions. Traditionally, the GPU is only used to accelerate certain parts of the graphics pipeline such as warping operations. General-purpose computing on graphics processing units (GPGPU) is the technique of using a GPU to perform computations that are usually handled by the CPU. The addition of programmable stages and high precision arithmetic allow developers to use stream processing of general data. Further author information: Send correspondence to Miguel Bordallo L´opez E-mail: miguelbl@ee.oulu.fi, Telephone: +358 449170541