Differential Spatiogram for People Re-Identification Cardellicchio A. 1 , Politi T. 1 , D’Orazio T. 2 , Ren ´ o V. 2 1 Politecnico di Bari 2 CNR ISSIA a.cardellicchio@gmail.com Abstract People re-identification is a crucial task in modern video surveillance systems, and poses a certain number of challenges related to low resolution, different views and illumination changes. We introduce the concept of Differential Spatiogram, a mathematical structure which enfolds informations about chro- matic relationships between each couple of images extracted from a gallery set and a probe set. The term spatiogram is used here because these mathematical structures are derived from color histograms and enfold spatial informations. Methodology Description Motivation Various approaches have been proposed to overcome the problem of people re-identification, each one taking in account different features related to color and textures of an image. Since the majority of available datasets contain low resolution images, color features like histograms [1] are more efficient than texture-based features like Recurrent High Structured Patches (RHSP) [2]. We propose a feature which enfold both relationships between chromatic components of an image (intra similarity) and between chromatic components of a pair of images (inter similar- ity). This feature, which we name Differential Spatiogram (DS), retains spatial information divid- ing images in n horizontal strips, in a similar way to [3], and it is generic as it can be used with any color space and similarity metric in order to improve its robustness to illumination and view changes. Algorithm The following pseudo-code describes the computation of DS for a given image dataset. Consider a image dataset D with cardinality 2c. 1. Split D into a gallery set G and a probe set P. Both G and P have cardinality c. 2. Divide each image in n horizontal strips. 3. For each strip of each image, extract color histogram. 4. For each couple of images [I a ∈ G, I b ∈ P ], compute Color Strip Similarity (CSS). 5. For each image [I ∈ G ∪ P ], compute Intra Image Similarity (IIS). 6. For each couple of images [I a ∈ G, I b ∈ P ], combine CSS and IIS to compute the Differential Spatiogram (DS). Features Here we describe the aforementioned features. Color Strip Similarity Color Strip Similarity is representative of inter similarity, and is computed as: CSS a,b =   D a 1 ,b 1 . . . D a n ,b n   where D a i b i is a similarity metric between histograms relatives to the i-th strip of images I a and I b respectively. Note the number of operations needed to compute the CSS feature is O(c 2 ). Intra Image Similarity Intra Image Similarity is representative of intra similarity, and is computed as: IIS a =     0 D a 1 ,a 2 ... D a 1 ,a n 0 0 . . . . . . . . . . . . D a n-1 ,a n 0 ... 0     where D a i a j is a similarity metric between histograms relatives to the i-th and the j-th strip of image I a . Note the number of operations needed to compute the IIS feature is O(nC ). Differential Spatiogram Differential Spatigram is computed as: DS ab =     CSS a 1 ,b 1 D 1,2 ... D 1,n 0 CSS a 2 ,b 2 . . . . . . . . . . . . . . . D n-1,n 0 ... 0 CSS a n ,b n     where CSS a i ,b i is the i-th component of CSS a,b and D i,j is: D i,j =  (IIS a i ,a j - IIS b j ,b i ) 2 (CSS a i ,b i CSS a j ,b j ) Considerations and results The mathematical structure of DS gives a certain number of advantages, both in terms of computational complexity (which is considerably lower than features like RHSP) and alge- brical properties. In fact, it can be shown that CSS is related to the eigenvalues of the corrisponding Differential Spatiogram as: T race(DS ab )= n  i=1 CSS a i ,b i = n  i=1 λ i Det(DS ab )= n  i=1 CSS a i ,b i = n  i=1 λ i where λ i is the i-th eigenvalue of DS ab . That gives the possibility to extract several algebric metrics from DS, including Trace and Det. DS can be used in three ways: 1. As a similarity metric between a couple of images. 2. To compute score matrices which take in account intra and inter relationships be- tween images. 3. As part of classification modules of an automatic surveillance system. Figure 1 shows the Cumulative Matching Characteristic (CMC) curve obtained using only Trace and Det metrics. Note these metrics have been computed both in RGB and HSV color spaces to show its flexibility. Figure 2 shows matching results using only Trace metric in HS color space for the VIPeR dataset. Figure 1: CMC curve for the Trace metric using HS, HSV and RGB color spaces and various similarity metrics. Best results are with HS and Bhattacharyya metric, worst with RGB and Euclidean distance. Figure 2: Matching using the Trace metric in HS color space for the VIPeR dataset. The first column on the left shows the probe image, the second column on the left shows the masked probe image and, to the right of it, the 20 top-ranked candidates are displayed. Highlighted images represent the correct match. The last two rows are examples of queries which do not have any correct match in the 10 top-ranked candidates. Conclusions and future works • Differential Spatiograms can be used as a robust and flexible similarity metric between images. • Differential Spatiograms can be used into the classification module of automatic video surveillance systems. • The number of operations needed to compute Differential Spatiogramsis considerably lower than the one needed to compute other color features. • Further research are needed to integrate Differential Spatiograms with other chro- matic and texture based features. References [1] M. Swain and D. Ballard, “Indexing via color histograms,” in Computer Vision, 1990. Pro- ceedings, Third International Conference on, pp. 390–393, Dec 1990. [2] M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani, “Person re-identification by symmetry-driven accumulation of local features,” in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pp. 2360–2367, June 2010. [3]D.-N. T. Cong, C. Achard, and L. Khoudour, “People re-identification by classification of silhouettes based on sparse representation,” in Image Processing Theory Tools and Applica- tions (IPTA), 2010 2nd International Conference on, pp. 60–65, July 2010.