In Proceedings of the 2001 International Conference on Tools With Artificial Intelligence, ICTAI-2001, Dallas, Texas, November 2001. Automatic Knowledge Acquisition from Subject Matter Experts Mihai Boicu, Gheorghe Tecuci, Bogdan Stanescu, Dorin Marcu, and Cristina Cascaval Learning Agents Laboratory, Department of Computer Science, MS 4A5 George Mason University, 4400 University Drive, Fairfax, VA 22030-4444 {mboicu, tecuci, bstanesc, dmarcu, ccascava}@gmu.edu, http://lalab.gmu.edu Abstract This paper presents current results in developing a practical approach, methodology and tool, for the development of knowledge bases and agents by subject matter experts, with limited assistance from knowledge engineers. This approach is based on mixed-initiative reasoning that integrates the complementary knowledge and reasoning styles of a subject matter expert and a learning agent, and on a division of responsibilities for those elements of knowledge engineering for which they have the most aptitude. The approach was evaluated at the US Army War College, demonstrating very good results and a high potential for overcoming the knowledge acquisition bottleneck. 1. Introduction This paper addresses the knowledge acquisition bottleneck in the development of knowledge bases and agents, bottleneck that we consider to be one of the main barriers in the generalized application of Artificial Intelligence. Traditionally, a knowledge-based system is built by a knowledge engineer (KE) who has to acquire the knowledge from a subject matter expert (SME) and to encode it into the knowledge base (KB). This is a very difficult process because the experts express their knowledge informally, using natural language, visual representations and common sense, often omitting many essential details that are considered obvious. In order to properly understand an expert’s problem solving knowledge and to represent it in a formal, precise, and complete knowledge base, the knowledge engineer needs to become himself a kind of subject matter expert. Therefore this process is very difficult, error-prone, and time-consuming [2]. Our research goal is to develop a theory, methodology and tool that will allow SMEs that do not have prior knowledge engineering experience to build knowledge-based systems by themselves, with no or very limited assistance from knowledge engineers. Our approach to this problem consists of developing a very capable learning agent shell that can perform many of the functions of a KE. The SME and the agent engage into a mixed-initiative process of developing the agent’s KB to incorporate the expertise of the SME. The concept of learning agent shell is an extension of the concept of expert system shell [5]. As an expert system shell, it includes a general inference engine that can be reused for multiple applications. In addition, it includes a general learning engine for building a knowledge base consisting of an object ontology that describes the entities from an application domain, and a set of problem solving rules expressed with these objects. The process of developing a knowledge-based system for a specific application relies on importing ontological knowledge from existing knowledge repositories, and on teaching the learning agent how to perform various tasks, in a way that resembles how an expert would teach a human apprentice when solving problems in cooperation. Over the years we have developed a series of learning agent shells from the Disciple family [8], most recently as part of the “High Performance Knowledge Bases” and “Rapid Knowledge Formation” programs supported by DARPA and AFOSR [3]. These programs emphasize the use of the challenge problems to focus the research and development efforts and measure the effectiveness of alternative technical approaches to the development of knowledge-based systems. Each of the following challenge problems required the rapid development and maintenance of a KB for a different type of application, and led to the development of an extended and improved Disciple system: 1) The Workaround challenge problem - planning how a convoy of military vehicles can circumvent or overcome obstacles in their path, such as damaged bridges. To solve this challenge problem we have developed the Disciple-Workaround learning agent, demonstrating that a knowledge engineer can rapidly teach Disciple, using military engineering manuals and sample solutions provided by a subject matter expert [6, 1, 9]. 2) The Course of Action (CoA) challenge problem - critiquing military courses of actions with respect to the principles of war and the tenets of army operations. To