Unsupervised learning of camera exposure control using randomly connected neural networks Oswald Berthold * Verena V. Hafner * * Humboldt-Universität zu Berlin, Dept. of Computer Science, Cognitive Robotics Group, Germany (e-mail: {oberthol|hafner} @ informatik.hu-berlin.de). Abstract: We use webcams on single board computers for vision-based control of flying robots. In that context we consider autonomous acquisition (bootstrapping) of exposure and gain control policies for the digital cameras. The policies are generated by neural networks with random connectivity which can be regarded as nonlinear expansion kernels acting on the input. We consider both feed-forward and recursive networks and apply these structures to learning the required policies. The camera represents an embodied robotic subsystem which is subject to temporal delays in its response. The performance measure is based on selective regions of interest in the image. The contribution of this paper is a complete embodied autonomous learning loop. Keywords: Robotics; Learning; Unsupervised; Neural Networks; Vision; MAV 1. INTRODUCTION Using a digital camera on embedded Single Board Com- puters (SBC) is a viable approach for realizing robotic vision on a flying robot as shown in Fig. 1(a). In order for vision algorithms to work, the distribution of the pixel values, as they reflect an outer contrast pattern, needs to be matched to the system’s sensitive operating range. The camera is a transducer, converting contrast patterns mediated by incident light into analog voltages and then into a list of digitally encoded numbers. There are three basic controls in the analog domain which act together on a common resulting variable, overall image brightness. These controls are the aperture, shutter- or integration time respectively and analog amplification, which are schemat- ically displayed in Figure 2. To avoid information loss during analog to digital (AD) conversion, the controls need to be set appropriately. Two conditions make the camera’s built-in Auto Exposure (AE) and Automatic Gain Control (AGC) mechanism unusable for our purposes. First, we would like to know the values of the control parameters at any time so they can be used as an efferent copy within a larger control system. Second, the Regions Of Interest (ROI) in the image, to which adjustment is referenced might be comprised of arbitrary subsets of sensor pixels, as is the case when using omnidirectional cameras as in Fig. 1(b) or cropping. This issue is treated comprehensively in Nourani-Vatani and Roberts (2007). We regard the camera control or constant image bright- ness problem as a simple and yet interesting instance of embodied robotic learning, as it is exemplary in terms of the closed sensorimotor loop by providing response latency ⋆ Oswald Berthold is funded by DFG RTG METRIK. and real-world noise. The complete system is pictured in Fig. 1(a). (a) Quadrotor flying robot (b) Omnidirectional view Fig. 1. Flying robot and omnidirectional camera view. Fig. 1(a) shows the complete system comprising of quadrotor airframe, onboard computing, sensors and omnidirectional lens mounted on top. Fig. 1(b) shows how the lens projects onto the rectangular image sensor. More than half of all pixels can be disregarded when calculating camera exposure settings because they are always black. 1.1 Related work AE and AGC are integral parts in the inventory of con- sumer and professional photography and video capture de- vices. Mostly, these solutions build on large datasets from which lookup tables for control values are extracted. Re- cently, techniques such as multi-slope cameras, bracketing and High-Dynamic-Range (HDR) photography brought additional challenges to exposure and focus control. These include the need for localized reference in the image, according to where ROIs, for example human faces, are detected to realize optimal exposure in those particular areas. Several solutions for robotic exposure control based on conventional techniques have been proposed. In Nourani-