Trajectory association and fusion across partially overlapping cameras Nadeem Anjum and Andrea Cavallaro * Queen Mary University of London Multimedia and Vision Group Mile End Road, E1 4NS London (United Kingdom) {nadeem.anjum, andrea.cavallaro}@elec.qmul.ac.uk Abstract We present a novel unsupervised inter-camera trajec- tory correspondence algorithm that does not require prior knowledge of the camera placement. The approach con- sists of three steps, namely association, fusion and link- age. For association, local trajectory pairs correspond- ing to the same physical object are estimated using mul- tiple spatio-temporal features on a common ground-plane. To disambiguate spurious associations, we employ a hybrid approach that utilizes the matching results on the image- and ground-plane. The trajectory segments after associa- tion are fused by adaptive averaging. Finally, linkage inte- grates segments and generates a single trajectory of an ob- ject across the entire observed area. We evaluated the per- formance of the proposed approach on a simulated and two real scenarios with simultaneous moving objects observed by multiple cameras and compared it with state-of-the-art algorithms. Convincing results are observed in favor of the proposed approach. 1. Introduction The reconstruction of objects’ trajectories across cam- eras facilitates the recognition of global behaviors for large scale events in applications such as sports analysis, remote sensing and video surveillance. This requires a mechanism for associating and integrating partially observed data in each camera view. Local trajectory information from in- dividual cameras may be corrupted by inaccuracies due to noise, objects re-entrances, occlusions and by errors due to crowded scenes. Therefore, trajectory association becomes a difficult task under such complex scenarios. In this paper, we consider the problem of object as- sociation across partially overlapping cameras using local * This work was supported in part by the EU, under the FP7 project APIDIS (ICT-216023). trajectories. Existing works perform association either on image-plane [7] or on ground-plane [4]. As image-plane trajectories are heavily affected by the perspective defor- mations, which cause inaccurate associations especially if the trajectories are far from cameras. On the other hand, accurate associations on ground-plane are hampered by the image- to ground-plane projections, which do not ensure unique association of an object trajectories observed in mul- tiple cameras. We propose a hybrid approach that combines the strength of both image- and ground-plane associations. Initial correspondence among trajectories is established on ground-plane using multiple spatio-temporal features and then image-plane reprojections of the matched trajectories are employed to resolve conflicting situations. This makes sure that only one trajectory of an object from each camera is associated to other cameras. The fusion is then applied to combine matched trajectories. A spatio-temporal linkage procedure connects the fused segments in order to obtain the complete global trajectories across the distributed set- up. Figure 1 shows the proposed flow diagram. The rest of the paper is organized as follows: Sec. 2 covers prior works in the field of object correspondence across multiple cameras. Section 3 provides the detailed description of the global ground-plane trajectories construc- tion from local image-plane segments. Section 4 covers the experimental results and finally Sec. 5 draws conclusions. 2. Prior work We categorize object correspondence approaches into supervised and unsupervised algorithms. Supervised tech- niques depends either upon the information contained in training samples or supplied manually by users. Several authors have proposed supervised association approaches such as Kettnaker et al. [6], Huang et al. [3], Dick et al. [1] and Wang et al. [9]. Unlike supervised techniques, unsuper- vised techniques do not require training samples or manual selection of the parameters. Recent unsupervised target as- 1 IEEE INTERNATIONAL CONFERENCE ON ADVANCED SIGNAL AND VIDEO BASED SURVEILLANCE (AVSS), GENOVA, ITALY, 2-4 SEPTEMBER 2009