1288 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 15, NO. 3, JUNE 2014 An Exploratory Study of Two Efficient Approaches for the Sensitivity Analysis of Computationally Expensive Traffic Simulation Models Qiao Ge, Biagio Ciuffo, and Monica Menendez Abstract—One of the main challenges arising when calibrating a complex traffic simulation model concerns the selection of the most important input parameters. The quasi-optimized trajectory- based elementary effects (quasi-OTEE) and the Kriging-based sensitivity analysis (SA) are two recently developed efficient ap- proaches for the SA of computationally expensive simulation mod- els. In this paper, two experimental studies using two different traffic simulation models (i.e., Aimsun and VISSIM) are presented to compare these two approaches and to better understand their advantages and disadvantages. Results show that both approaches are able to identify, to a good degree, the important parameters. In particular, the quasi-OTEE is better for screening the parameters, whereas the Kriging-based SA has higher precision in ranking the parameters. These findings suggest the following rule of thumb for the SA of computationally expensive traffic simulation models: the quasi-OTEE SA can be used first to screen the parameters and to decide which parameters to discard. Then, the Kriging-based SA can be used to refine the analysis and calculate first-order indexes to identify the correct rank of the important parameters. Index Terms—Calibration, sensitivity analysis (SA), traffic sim- ulation model, uncertainty. I. I NTRODUCTION T RAFFIC simulations have become indispensable tools for academicians and practitioners worldwide. The reliability of the results achieved using a traffic simulation model is strictly connected with the ability to calibrate the model. The calibration is vital, yet it can be rather difficult when the traffic model involved is computationally expensive. Due to the limitation of time and other resources, when deal- ing with a computationally expensive model, a widely adopted approach is to select a limited number of input parameters rather than the whole parameter set for calibration. However, there is usually no formal procedure for selecting these param- eters, other than choosing the ones that appear to the model user as most likely to have a significant effect on the result Manuscript received July 15, 2013; revised November 14, 2013 and January 19, 2014; accepted March 1, 2014. Date of publication April 25, 2014; date of current version May 30, 2014. This work was supported by the European Cooperation in Science and Technology (COST) under COST Action TU0903: “Methods and tool for supporting the use, calibration, and validation of traffic simulations models” (MULTITUDE). The Associate Editor for this paper was M. Brackstone. Q. Ge and M. Menendez are with the Institute for Transport Planning and Systems, Swiss Federal Institute of Technology (ETH Zurich), Zurich 8093, Switzerland (e-mail: qiao.ge@ivt.baug.ethz.ch; monica.menendez@ivt. baug.ethz.ch). B. Ciuffo is with the Institute for Energy and Transport of the European Commission, Joint Research Center, Ispra, Italy (e-mail: biagio.ciuffo@jrc. ec.europa.eu). Digital Object Identifier 10.1109/TITS.2014.2311161 (such criteria is often dictated by former experiences). As one could imagine, the selection of an incomplete set of parameters for calibration may lead to multiple issues, including but not limited to, unrealistic values for the calibrated parameters, and inaccuracy and bias of the model outputs. To avoid the potential cascading effects caused by subjectively choosing the incomplete or wrong set of calibration parameters, a sensitivity analysis (SA) of the input parameters is essentially required. The SA explores the relationship between the simulation output and input parameters [1]. A proper SA, including the initial screening of parameters, can provide both quantitative and qualitative information regarding the effects of different model parameters (and their variations) on the simulation re- sults. Therefore, a good SA (particularly those global SA in [1]), can be very valuable for the subsequent model calibration. The elimination of unimportant parameters via an efficient SA may not only reduce the total efforts spent in the actual model calibration but also enhance the stability and quality of the calibration. Furthermore, a global SA is able to provide the model sensitivity information in the whole input space rather than around some specific points. This feature can bring further economy to the calibration as the SA results can be reused at anytime when the same model is calibrated. In spite of the importance of the SA described, a proper SA for traffic models is barely performed in common practice. In fact, the survey carried out within the COST Action TU0903 [2] indicates that 31% of VISSIM users, 43% of Aimsun users, and 24% of Paramics users failed to perform the SA in their previ- ous work or applications. Among the few examples found in the literature, the one-at-a-time (OAT) approach and the variance- based approach [1] appear to be two most used approaches. The OAT approach was adopted in [3]–[5] to identify the important parameters in VISSIM for calibration and in [6] for calibrating the intelligent driver model and the velocity difference model. The variance-based approach was employed in the calibration of VISSIM in [7]–[10], of Aimsun in [11]–[13], of Paramics in [14] and [15], and of the two car-following models in [16]. However, both approaches have certain limitations in the application. The OAT approach has very high computational efficiency, but it is indeed a “local” SA approach and thus fails to detect the interaction effects among parameters [1]. On the other hand, although the variance-based approach is able to catch the interaction effects, it often requires a large number of model evaluations [1]. Hence, it becomes unfeasible when the model has many parameters and the computational cost of simulation is expensive. 1524-9050 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.