Received: 14 March 2017 Revised: 5 June 2017 Accepted: 1 September 2017 DOI: 10.1111/exsy.12249 SPECIAL ISSUE PAPER Panorama construction for PTZ camera surveillance with the neural gas network Karl Thurnhofer-Hemsi Ezequiel López-Rubio Enrique Domínguez Rafael Marcos Luque-Baena Miguel A. Molina-Cabello Department of Computer Languages and Computer Science, University of Málaga, Bulevar Louis Pasteur 35, 29071 Málaga, Correspondence Enrique Domínguez, Department of Computer Languages and Computer Science, University of Málaga, Bulevar Louis Pasteur 35, 29071 Málaga, Spain. Email: enriqued@lcc.uma.es Funding information Ministry of Economy and Competitiveness of Spain, Grant/Award Number: TIN2014-53465-R, TIN2014-57341-R and TIN2016-75097-P; Autonomous Government of Andalusia (Spain, Grant/Award Number: TIC-6213, TIC-657 and FPU15/06512; Spanish Ministry of Education, Culture and Sport under the FPU program Abstract The construction of a model of the background of a scene still remains as a challenging task in video surveillance systems, in particular for moving cameras. This work presents a novel approach for constructing a panoramic background model based on the neural gas network and a subse- quent piecewise linear interpolation by Delaunay triangulation. Furthermore, an ensemble model of neural gas networks is also proposed. The approach can handle arbitrary camera directions and zooms for a pan-tilt-zoom camera-based surveillance system. After testing the proposed approach on several indoor sequences, the results demonstrate that the proposed methods are effective and suitable to use for real-time video surveillance applications. KEYWORDS background modeling, PTZ cameras, panorama construction, neural gas network 1 INTRODUCTION Pan–tilt–zoom (PTZ) cameras have become increasingly popular in monitoring public areas (Chung-Chen Chen, Yi Yao, Drira, Koschan, & Abidi, 2009; Ding, Song, Morye, Farrell, & Roy-Chowdhury, 2012; Konda, Conci, & De Natale, 2016) due to their high mobility and zoom capability. Although omnidirectional cameras are promising candidates for monitoring multiple latent activities in the area of interest (Boult, Gao, Micheals, & Eckmann, 2004), this kind of cameras have non-uniform resolution and are unable to provide close observations of particular targets. In these cases, where PTZ cameras are needed, the combination of these two types of cameras (omnidirectional and PTZ) is proposed in order to facilitate a continuous monitoring of the whole surveillance area and detailed observations of specific targets simultaneously (Chen et al. 2008). Nevertheless, this dual-camera system may be still an expensive and complex solution in some scenarios. For this reason, in this paper, we are focusing on an active sensing approach to multiple object detection and tracking using a single PTZ camera. For the images/videos captured by static cameras, the most common and efficient approach to moving object detection is background subtraction, that consists in maintaining an up-to-date model of the fixed background and detecting moving objects as those that deviate from such model. Compared to other approaches, such as optical flow, this approach is computationally affordable for real-time applications, is independent of moving object velocity, and is not subject to the foreground aperture problem. However, traditional background subtraction algorithms assume the cameras are static, and this leads to false detection when the camera moves (Kim, Yun, Yi, Kim, & Choi, 2013; Xue, Liu, Ogunmakin, Chen, & Zhang, 2013). Due to this camera movement, even pixels belonging to static objects appear to move in the camera frame (called ego-motion effect). Extensive research has been carried out regarding object detection for moving cameras. Some proposal are based on the optical flow cluster- ing that consists in calculating dense or sparse optical flows and clustering them to identify moving object regions (Varcheie & Bilodeau, 2011). Another methods are based on the estimation of the transformation parameters between consecutive frames (López-Rubio & López-Rubio, 2015). Our approach is based on mosaicing the background (Azzari & Bevilacqua, 2006; Bevilacqua & Azzari, 2006) that consist in creating a mosaiced or panoramic background image and then using a background subtraction technique to extract moving object regions. In this paper, we address the problem of moving-objects detection for PTZ cameras and propose a method based on building a panoramic back- ground model using a type of competitive neural network. Apart of the traditional and frequently cited seminal papers related to competitive learning (Ahalt, Krishnamurthy, Chen, & Melton, 1990; Uchiyama & Arbib, 1994; Yair, 1992), recent successful applications in the computer vision field can be found in the literature (Chen, Shen, & Long, 2016; Ozan, Kiranyaz, & Gabbouj, 2016; Valente, & Abrao, 2016; Xie et al., 2016). In our Expert Systems. 2017;e12249. wileyonlinelibrary.com/journal/exsy Copyright © 2017 John Wiley & Sons, Ltd. 1 of 13 https://doi.org/10.1111/exsy.12249