Elucidative fusion systems ± an exposition Belur V. Dasarathy * Dynetics, Inc., P.O. Box 5500, Huntsville, AL 35814-5500, USA Received 9 January 1999; received in revised form 21 March 2000; accepted 18 April 2000 Abstract Much of the research in the area of multi-source fusion systems reported in the literature has concentrated on alternative methods of accomplishing fusion of information, such that the resulting fused output is in some sense better than any of the inputs indi- vidually by themselves. There is a wealth of literature expounding many variations on this theme of improving the information quality, reliability, or robustness through the use of a plethora of fusion concepts and tools. However, there has been little reported research aimed at providing an understanding of the causal relationship between the input information and the resulting fused output. This study explores this mostly virgin territory by conceptualizing fusion systems that are elucidative, i.e., systems that can, in some fashion, explain the results of the fusion process in terms of, for example, the relative in¯uence of the dierent input in- formation components (from the dierent sources) on the fused result. This new concept of elucidative fusion systems is illustrated in this study by inculcating such an elucidative property into a class of fusion systems operating on the principles of case-based reasoning. The potential for application to real-world problems is also demonstrated using the example of an audio-video system for recognition of spoken French vowels. Ó 2000 Elsevier Science B.V. All rights reserved. Keywords: Information fusion; Elucidative fusion system; Case-based reasoning; Audio-video system; Vowel recognition 1. Elucidative fusion systems ± what, why, and how? The American heritage dictionary de®nes ÔelucidateÕ, as Ôto make clear, clarify or explainÕ which in turn is further elaborated as Ôto oer reasons forÕ. Elucidate, to quote directly from the dictionary [1], Ôis used most appropriately in contexts that suggest an attempt to throw light in some way on a complex subjectÕ. The reasons why a speci®c decision is made in any given instance by a fusion system operating on information acquired from disparate sources are indeed complex and not easily understood just by looking at the resulting decisions. Elucidative fusion systems are accordingly de®ned as systems that have the ability to oer, in ad- dition to the decisions they develop through multi- source information fusion, the reasons or drivers for such decisions. They also oer a measure of in¯uence of the dierent information sources on the fused decision. Multi-source, multi-sensor information fusion-based decision systems have begun to appear in many appli- cation areas, including defense, aerospace, robotics, medical diagnostics, security systems, ®nancial market analysis and the like. Deployment of such automated decision systems in many of these application scenarios is often hindered by the Ôfear of the unknownÕ on the part of the user communities and their ultimate human decision-makers. An useful innovation, which would be comforting to the end-user, is a system that not only oers a decision but also provides an explanation as to why a speci®c decision is made. (It is to be noted here that the word ÔexplanationÕ as employed here has a slightly dierent ¯avor than the way it is used in the strict context of case-based reasoning.) In other words, one would like to have a system that can elucidate its decisions and identify the decision drivers so that the user can develop ÔtrustÕ in the performance of the sys- tem. It would be desirable for the user to, not only ob- tain the decisions from the fusion system, but also understand the rationale underlying the decisions de- rived from such automated multi-source information fusion systems. Any feedback that can be provided to the user in this regard will serve many purposes. For example, it can reinforce faith in the reliability and ro- bustness of the decision systems. It can also provide an opportunity for the user to have a sanity-check on the decisions provided by the system. This understanding of the ÔwhyÕ of the decisions de- rived by the fusion system can be expressed qualitatively Information Fusion 1 (2000) 5±15 www.elsevier.nl/locate/inus * Tel.: +1-256-964-4355; fax: +1-256-922-9260. E-mail address: belur.d@dynetics.com (B.V. Dasarathy). 1566-2535/00/$ - see front matter Ó 2000 Elsevier Science B.V. All rights reserved. PII: S 1 5 6 6 - 2 5 3 5 ( 0 0 ) 0 0 0 0 6 - 3