Decision Analysis Techniques for Knowledge Acquisition: Combining Information and Preferences using Aquinas Jeffrey M. Bradshaw and John H. Boose Knowledge Systems Laboratory, Advanced Technology Center, Boeing Computer Services P.O. Box 24346, M/S 7L-64, Seattle, Washington 98124 USA (206) 865-3422 ABSTRACT The field of decision analysis is concerned with the application of formal theories of probability and utility to the guidance of action. Decision analysis has been used for many years as a way to gain insight regarding decisions that involve significant amounts of uncertain information and complex preference issues, but it has been largely overlooked by knowledge-based system researchers. This paper illustrates the value of incorporating decision analysis insights and techniques into the knowledge acquisition and decision making process. This approach is being implemented within Aquinas, an automated knowledge acquisition and decision support tool based on personal construct theory that is under development at Boeing Computer Services. The need for explicit preference models in knowledge-based systems will be shown. The modeling of problems will be viewed from the perspectives of decision analysis and personal construct theory. We will outline the approach of Aquinas and then present an example that illustrates how preferences can be used to guide the knowledge acquisition process and the selection of alternatives in decision making. Techniques for combining supervised and unsupervised inductive learning from data with expert judgment, and integration of knowledge and inference methods at varying levels of precision will be presented. Personal construct theory and decision theory are shown to be complementary: the former provides a plausible account of the dynamics of model formulation and revision, while the latter provides a consistent framework for model evaluation. Applied personal construct theory (in the form of tools such as Aquinas) and applied decision theory (in the form of decision analysis) are moving along convergent paths. We see the approach in this paper as the first step toward a full integration of insights from the two disciplines and their respective repertory grid and influence diagram representations. 1 THE CASE FOR EXPLICIT MODELING OF PREFERENCES 1.1 PREFERENCE MODELING AND KNOWLEDGE-BASED SYSTEMS Many knowledge-based systems are prescriptive in nature. They aim not only to describe situations but also to recommend specific actions. Recommendations made by such systems depend on: the alternatives available, information about consequences associated with the alternatives, and preferences among these consequences. Research in building knowledge-based systems has typically focused on the first two considerations — many approaches have been proposed for structuring and eliciting alternatives and for modeling potentially uncertain information. By contrast, relatively little effort has been made in the knowledge acquisition community toward understanding how to explicitly represent and quantify preferences. The neglect of this issue limits the effectiveness of knowledge-based systems for many types of problems. Example: Preferences in MYCIN. Knowledge- based systems typically treat preferences implicitly and heuristically, making no provision for value structures differing from those built into the system. Several recent papers have discussed the need for explicit preference models to be included in the knowledge engineering process (e.g., Henrion, 1987; Henrion & Cooley, 1987; Holtzman, 1989; Horvitz et al., 1988; Keeney, 1986a; Langlotz, Shortliffe, & Fagan, 1986). In their discussion of preferences, Langlotz, et al. (1986) cite an example rule from MYCIN (see Figure 1). This heuristic captures a physician's knowledge that tetracycline therapy should be avoided for children because it may cause dental staining. If: 1) The therapy under consideration is tetracycline 2) The age (in years) of the patient is less than 8 Then: There is strongly suggestive evidence (.8) that tetracycline is not an appropriate therapy for use against the organism. Figure 1: The MYCIN tetracycline heuristic, slightly simplified for illustration purposes. Clancey (1983) gives a possible chain of four support rules for this heuristic (Figure 2). The first three inferences have to do with how one event is related to