Proceedings of DETC’08 ASME 2008 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference New York City, New York, August 3-6, 2008 DETC2008/DAC-49669 A COMPREHENSIVE METRIC FOR COMPARING TIME HISTORIES IN VALIDATION OF SIMULATION MODELS WITH EMPHASIS ON VEHICLE SAFETY APPLICATIONS H. Sarin, M. Kokkolaras ∗ , G. Hulbert, P. Papalambros {harshit,mk,hulbert,pyp}@umich.edu Department of Mechanical Engineering The University of Michigan, Ann Arbor S. Barbat, R.-J. Yang {sbarbat,ryang}@ford.com Passive Safety, Research and Advanced Engineering Ford Motor Company, Dearborn, MI ABSTRACT Computer modeling and simulation are the cornerstones of product design and development in the automotive industry. Computer-aided engineering tools have improved to the extent that virtual testing may lead to significant reduction in prototype building and testing of vehicle designs. In order to make this a reality, we need to assess our confidence in the predictive ca- pabilities of simulation models. As a first step in this direction, this paper deals with developing a metric to compare time histo- ries that are outputs of simulation models to time histories from experimental tests with emphasis on vehicle safety applications. We focus on quantifying discrepancy between time histories as the latter constitute the predominant form of responses of inter- est in vehicle safety considerations. First we evaluate popular measures used to quantify discrepancy between time histories in fields such as statistics, computational mechanics, signal pro- cessing, and data mining. Then we propose a structured combi- nation of some of these measures and define a comprehensive metric that encapsulates the important aspects of time history comparison. The new metric classifies error components asso- ciated with three physically meaningful characteristics (phase, magnitude and topology), and utilizes norms, cross-correlation measures and algorithms such as dynamic time warping to quan- tify discrepancies. Two case studies demonstrate that the pro- posed metric seems to be more consistent than existing metrics. It is also shown how the metric can be used in conjunction with ratings from subject matter experts to build regression-based val- ∗ Corresponding author, Phone/Fax: (734) 615-8991/647-8403 idation models. 1 Introduction Vehicle safety has become a major concern in modern so- ciety. Automotive manufacturers have to meet several regula- tions and mandatory Federal Motor Vehicle Safety Standards (FMVSS). Additionally, consumer information programs such as the New Car Assessment Program (NCAP) and the Insurance Institute of Highway Safety (IIHS) impose further requirements for vehicle safety. Currently, testing whether these requirements are satisfied is conducted through numerous, costly and time- consuming physical experiments. Computer modeling and simulation-based methods for vir- tual vehicle safety analysis and design verification could make this process more cost-efficient. Moreover, virtual testing (VT) can improve real-world vehicle safety beyond regulatory require- ments since computer predictions can be used to extend the range of protection to real-world crash conditions at speeds and config- urations not addressed by current regulations. To achieve the promises of VT, computer predictions need verification and validation (V&V), so that the designs obtained using simulation models can be cleared for production with min- imized prototype testing. The AIAA guide for verification and validation of computational fluid dynamics simulations defines verification and validation as follows [1]. Verification is the process of determining that a model implemen- tation accurately represents the developer’s conceptual descrip- 1 Copyright  2008 by ASME