Roc curves for performance evaluation of video sequences processing systems for surveillance applications F. Oberti, A. Teschioni and C.S. Regazzoni DIBE – University of Genoa – Via Opera Pia 11a, I-16145 GENOA (Italy) oberti@dibe.unige.it carlo@dibe.unige.it Abstract Performances evaluation of image processing intermediate results in video based surveillance systems is extremely important due to the variety of approaches to this task. In this paper, an approach based on the use of Receiver Operating Characteristics (ROC) curves in order to evaluate the performances of a vision complex system for surveillance purposes is presented. ROC curves have already been used in other research fields as comparison of edge detection algorithms or evaluation of artificial neural networks: in this case they are used in order to compare different parameters selections within a system for the localization of moving objects. Presented results show the possibility of using ROC curves as a mean for evaluation and comparison of video-based surveillance systems. 1. Introduction Thanks to the increasing development of complex vision systems, it becomes strictly necessary to introduce quantitative performance evaluation methods. Such methods should make it possible both comparing results provided by different surveillance systems and selecting optimal parameters for each one, depending on the specific functionality of a system and on the particular characteristics of the monitored environment. In this paper, we show that Receiver Operating Characteristics [1] (ROC) curves provide a well assessed tool that can be used for the above purpose. Video-based Surveillance systems are complex systems performing different functionalities, depending on the working application field; nevertheless, target detection can be indicated as a general functionality of a surveillance system. Consequently, false alarm and correct detection rates can be defined univocally for a quite large class of problems. On this basis, performances of different system modules can be assessed in terms of their capability to produce intermediate results potentially capable of satisfying the final goal. ROC curves are a performance evaluation tool based on the representation on a single diagram of the joint behavior of false alarm and correct detection rates obtained by a given system. ROC curves have been introduced and extensively used in decision theory related to signal processing applications in the communications field. A ROC curve is generally obtained by computing pairs (PD,PF) in correspondence of different values of some parameter that regulates the behavior of the decision module of the system, usually a threshold η in the decision space related to a binary hypothesis testing problem (see Fig.1). Figure 1: Example of typical ROC curves However, more recently, ROC curves have been used in evaluation of computer vision tasks: comparison of edge detection algorithms [2] or evaluation of artificial neural networks for medical imaging [3] are two examples. In this context, they have mainly been applied to tasks dealing with not highly structured problems. Nevertheless, it has emerged that the high dimensionality of the decision process in vision tasks imply a not straightforward extension of hypothesis testing approaches for mono dimensional signals. This work aims at presenting a new approach to the use of ROC curves for performances evaluation of complex vision systems: in particular, a quantitative methodology is proposed for estimation of false alarm and correct detection rates from intermediate results of image processing modules of surveillance systems. The main problem here faced is to find a meaningful way of describing system performances in presence of an