1 Simulating Human Decision Making for Testing Soft and Hard/Soft Fusion Algorithms Donald J Bucci, Sayandeep Acharya, and Moshe Kam Department of Electrical and Computer Engineering Drexel University Philadelphia, Pennsylvania, 19104-2816 Abstract—Current methods for evaluating the effects of human opinions in data fusion systems are often dependent on human testing (which is logistically hard and difficult to arrange for repeated tests of the same population). The alternative is to use hypothetical examples, which tend to be simplistic. To facilitate studies of data fusion architectures which integrate “soft” human- generated decisions, we have used a simulator of subjective beliefs. The simulator is based on the two-stage dynamic signal detection model of Pleskac and Busemeyer (2010). We use this scheme to simulate human opinions and combine them using belief fusion methods, including Bayes’ Rule; Dempster’s Rule of Combination (DRC); Yager’s rule; the Proportional Conflict Redistribution Rule #5 (PCR5) from Dezert-Smarandache the- ory; and the consensus operator from subjective logic. In our simulations, the DRC and Bayes rule exhibited performance that was on par with, and in some cases better than PCR5 and the consensus operator (when used in conjunction with a measure of source reliability). In all simulated cases, Yager’s rule exhibited inferior performance. I. I NTRODUCTION The integration of subjective data sources in a data fu- sion system (also known as “soft fusion”) has been studied intensely over the past few years [1]. In addition to the “hard fusion” of sensory information from devices that use electronic, optical, and acoustic modalities, there is a grow- ing interest in augmenting decision making with available “soft” sensors (i.e., human opinions and assessments [1]). The incorporation of human opinions into data fusion systems could potentially improve accuracy and reliability. However, human opinions are difficult to model as they do not exhibit fixed error probabilities, and are often not easily characterized by probability distributions [1]. Furthermore, the employment of large numbers of humans for testing can be logistically challenging and expensive, and opportunities to re-test the same humans on modified data presentations and exposition schemes is often difficult. At least in the early stages of testing and tuning of data fusion algorithms, it may be desirable to use models of human decision making rather than using actual human-generated data. Much work has been devoted over the past fifty years to developing new ways of representing and combining subjec- tive and imprecise beliefs in areas such as pattern recognition, biometrics, medical diagnostics, and autonomous navigation. However, the majority of studies which include elements of human decision making have resorted to hypothetical ex- amples, observing how fusion methodologies perform with respect to how a ”logical and coherent human” would reason [2]–[4]. Other studies have adopted simple stochastic models, observing the performance of combined decisions through Monte Carlo simulations (e.g., confusion matrices [5]). There have also been several attempts to amass groups of humans for direct testing and analysis [6]. This last approach is in many ways preferred to the models it has replaced, but is logistically cumbersome and somewhat inflexible, especially in assessing systems and algorithms that require the tuning of a large number of parameters. Models of human decision making from the social sciences and cognitive psychology have not been applied extensively to soft and hard/soft fusion systems. The few studies that have used such models tend to look at how task reward structures in- fluence human decision-making strategies in situations where the human element acts in a supervisory role [7]. The present study seeks to analyze soft fusion systems where subjective data sources provide confidence assessments of the decisions they make. In Section II, we overview the model for human simulation used here, known as two-stage dynamic signal detection (2DSD). In 2DSD, humans are modeled via a tuple of parameters which direct a stochastic process that represents an internal evidence accumulation between two outcomes. The human tuples used here are taken from [8], and are the result of modeling human responses in a line length discrimination task, where human subjects are shown different pairs of lines and asked to identify (1) the longer of the two lines and (2) rate their confidence in their response on a subjective probability scale. In Section III, we review a few methods for representing and combining subjective beliefs. In Section IV, we describe ways of mathematically formulating human opinions for use with the belief combination methods of Section III. Finally, in Section V we use a subset of the data from [8] to simulate human opinions for the line length discrimination task and fuse them using the methods described in Sections III and IV. II. HUMAN SIMULATION METHODOLOGY A. Two-Alternative Forced Choice Tasks Human decision-making models have been a topic of inter- est for psychologists since the early 1960s [9]. The majority of work has been addressing decision making in two-alternative forced choice (TAFC) tasks, in which a subject is presented with a scenario and is forced to choose between two alterna- tives [9]. Models of decision making based on TAFC tasks 978-1-4673-5239-0/13/$31.00 ©2013 IEEE