Traffic Sensory Data Classification by Quantifying Scenario Complexity ⋆ Jiajie Wang 1 , Chi Zhang 2 , Yuehu Liu 3 , Qilin Zhang 4 Abstract— For unmanned ground vehicle (UGV) off-line testing and performance evaluation, massive amount of traffic scenario data is often required. The annotations in current off-line traffic sensory dataset typically include I) types of roadways II) scene types III) specific characteristics that are generally considered challenging for cognitive algorithms. While such annotations are helpful in manual selection of data, they are insufficient for comprehensive and quantitate measurement of per-roadway-segment scenario complexity. To resolve such limitations, we propose a traffic sensory data classification paradigm based on quantifying the scenario complexity for each roadway segment, where such quantification is jointly based on road semantic complexity and traffic element complexity. The road semantic complexity is a proposed measurement of the complexity incurred by the static elements such as curvy roads, intersections, merges and splits, which is predicted with a Support Vector Regression (SVR). The traffic element complexity is a measurement of complexity due to dynamic traffic elements, such as nearby vehicles and pedestrians. Experimental results and a case study verify the efficacy of the proposed method. I. I NTRODUCTION During the developing, testing and verification cycle of unmanned grounded vehicle (UGV) system (e.g. [12]), a large amount of traffic scenario data is utilized for per- formance evaluation. In recent years, to meet the practical demand of autonomous driving technology, especially for the research and development on environmental cognition and understanding algorithms [20]–[22], many traffic scene datasets have been proposed, such as KITTI [5], RobotCar [10]. These datasets are usually collected in traffic scenarios with dynamic changes in cognition complexity, including different types of roads, scene contents and scene charac- teristics. However, the lack of quantitative characterization of the scene complexity in these datasets could impede interpretable evaluation of UGV systems. On the other hand, in the unmanned off-line testing [2], [7], we find that there is usually a negative correlation between the unmanned vehicle algorithm performance and scenario complexity. Traffic data with higher scenario complexity typically leads to worse performance of the environment cognition and understanding algorithm. If we use unorganized data to test and evalu- ate an environment-aware understanding algorithm for an ⋆ This work was supported by National Natural Science Foundation of China (Grant No: 91520301). 1 Jiajie Wang is with the school of Software Engineering, Xi’an Jiaotong University, Xi’an, China. 2 Chi Zhang is the corresponding author and is with the Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, China. colorzc@stu.xjtu.edu.cn 3 Yuehu Liu is with the Faculty of the Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, China. 4 Qilin Zhang is with HERE Technologies, Chicago, Illinois, 60606, USA. unmanned system, the results are mostly indistinguishable: an algorithm with 0.84 overall accuracy may consistently perform worse than a competing algorithm with only 0.83 overall accuracy in common road scene scenarios. Therefore, the complexity of the scene data needs to be incorporated for reliable evaluation of UGV systems. Hence we propose a method in quantifying scenario complexity to rank massive scene data. The complexity is calculated on the basis of the road types, scene types, challenging condition and traffic elements. Scenario com- plexity is computed from two perceptual data levels: 1) Road semantic complexity (RSC). We propose a road se- mantic complexity prediction method based on support vec- tor Regression (SVR). The road semantic complexity of a given non-hierarchical semantic descriptor is predicted by learning the relationship between the road label and the semantic descriptor. 2) Traffic element complexity (TEC). Traffic elements are moving entities that participate in road traffic activities. In this paper, vehicles are chosen to be the representative of the traffic elements. We devise description matrices of traffic elements and TEC calculation to quantify the complexity. The contributions of this paper are as follows. 1) Traffic sensory data is semantically quantified in terms of scenario complexity. 2) A comprehensive scenario complexity is formulated based on scene types, test sites/location information and dynamic traffic elements on a per-segment basis. This rest of the paper is organized as follows. Section II reviews existing datasets and their respective problems, and the formulation of scenario complexity. In the Section III, a new scene semantic feature is proposed, followed by the sensory data classification framework. Section IV introduces related applications, including accelerating off-line UGV evaluation and grading data synthesis. Section V summarizes and concludes the paper. II. RELATED WORK AND SYSTEM OVERVIEW Several road-sensing datasets for unmanned vehicle testing have been proposed since 2012, including KITTI [4], [5], RobotCar [10], Cityscape, Udacity, BDDV, etc. The KITTI dataset was proposed by researchers from Karlsruhe Institute of Technology and the Toyota Institute of Technology at Chicago, which is the largest multi-sensory 1 traffic scene 1 Including stereo RGB/grayscale cameras, LIDAR and GPS/IMU. Multi- sensory data, especially remote sensing LIDAR data [1], [18], [19], provides more discriminative information for object detection, classification and localization.