Knowledge Matters: Performance with Decision Support Michael Davern (mjdavern@unimelb.edu.au) The University of Melbourne Victoria 3010, Australia Arnold Kamis (akamis@bentley.edu) Bentley College Waltham, MA 02452 Background and Research Question. There has been a long tradition of experimental research exploring the behavioral aspects of effective decision support; from the well-known Minnesota experiments (Dickson, Senn et al. 1977) to the more recent series of studies by Todd and Benbasat (Todd & Benbasat 1992; Todd & Benbasat 1993; Todd & Benbasat 1994; Benbasat & Todd 1996; Todd & Benbasat 1999). Yet despite the volume of research, rather surprisingly, user domain knowledge has not been commonly featured explicitly as a central construct. In this study, we address the question of the effect of knowledge on performance with a decision tool in a preferential choice task. That user domain knowledge is important, even in a preferential choice task such as a consumer purchase decision, seems axiomatic. The nature of the effect of user domain knowledge is, however, very much an empirical question. For example, knowledge can be very helpful, improving the problem solving performance obtained for a given level of effort (Newell 1990), even without decision support. Alternatively, the domain knowledgeable user may be less mindful (Langer 1989), or more complacent and thereby fail to take full advantage of the functionality offered by the decision tool. The purpose of this study is to empirically explore the potentially divergent effects of knowledge. Specifically, this research asks the question: what is the effect of domain knowledge on performance with a decision tool in a preferential choice task? Drawing on prior work, we consider the influence of domain knowledge and effort on accuracy with two different decision tools varying in effort and knowledge requirements. Theory and Hypotheses. In a preferential choice task, a range of decision strategies of varying degrees of effort and accuracy have been observed and described in the behavioral decision theory literature. Decision makers recognize the trade-offs between effort and accuracy and select an appropriate strategy as an adaptive response to the goals and constraints they face (e.g., incentives for accuracy versus limited available time for strategy execution) (Payne et al. 1993, Johnson & Payne 1985). Extending the Effort-Accuracy research, Todd and Benbasat found that decision tools can add value by decreasing effort, given a desired level of accuracy (Todd and Benbasat 1992) or increasing the accuracy achieved given a certain level of effort (Todd and Benbasat 1999). The twin objectives, accuracy maximization and effort minimization, generate value to a user as he or she makes trade-offs with a decision support tool, although effort minimization tends to take precedence over accuracy maximization (Todd and Benbasat 1994; Benbasat and Todd 1996). The most important goal for decision support tools in this context is to reduce the effort required to execute a given strategy and thus make a more accurate strategy available for less effort, thereby improving performance. The Effort-Accuracy literature does not directly consider the role of domain knowledge. In the preferential choice task context, knowledge may include knowledge of attribute ranges, attribute trade-offs (of value in a compensatory strategy) or of which attribute best facilitates distinguishing between different alternatives (which could assist in compensatory strategies or in non-compensatory strategies, such as the lexicographic strategy). Knowledge may also be in the form of a specification of a stereotypical ideal (Fiske and Pavelchak 1986). Knowledge can substitute for effort (Newell 1990). Thus, any consideration of knowledge effects is potentially confounded if it does not also consider effort effects. Similarly, any study of effort is potentially confounded if it does not consider or appropriately control for knowledge. Knowledge and effort also complement each other; performance is enhanced when a decision maker relies on a combination of effort and knowledge rather than exclusively on one factor alone. As substitutes, a decision maker can compensate for a lack of knowledge with additional effort and still achieve a given level of performance. Conversely, a knowledgeable decision maker can achieve a given level of performance with less effort than a less knowledgeable individual. Newell (1990) elegantly characterizes this as a trade-off between prior knowledge and search effort in terms of “equi-performance isobars” (see figure adapted from Newell 1990). How does this effort-knowledge relationship come into play in the presence of decision support? Clearly, tools will vary with respect to the effort required, the accuracy they can achieve, and the requirement and opportunity for exploiting knowledge. As a control, we developed a tool (LEX) which facilitates a lexicographic strategy that is non-compensatory, low effort, low accuracy, and low in Prior Knowledge Search Effort Increasing Performance Equi-performance Isobars