Evaluating Support for Improvisation in Simulated Emergency Scenarios D. Mendonça G.E.G. Beroggi W.A. Wallace New Jersey Institute of Technology Spring Analytica Rensselaer Polytechnic Institute djm@njit.edu giampiero@beroggi.net wallaw@rpi.edu Abstract Technological systems involving hazards are typically managed by experienced personnel guided by well- formulated, pre-determined procedures. These procedures are designed to ensure that operations proceed in a safe and cost-effective manner. Yet normal operations in these systems are exposed to unexpected contingencies that can require personnel to develop and deploy new procedures in real-time. Creative thinking in such situations is therefore necessary in order to prevent degradation of operations, particularly when there is potential for personal injury, economic loss or environmental damage. One approach to addressing these situations is improvisation. The research described here discusses a series of studies conducted to evaluate the efficacy of a computer-based system for supporting improvisation in simulated crisis situations. The design and implementation of the system are first discussed, drawing upon prior work in blackboard-based systems. The experimental design is then reviewed, followed by a discussion of how the studies were run using groups of emergency response personnel from the Port of Rotterdam in The Netherlands. The group task was to address unexpected contingencies in a timely fashion. A number of measures of group decision effectiveness and uniqueness are presented. Results of the studies suggest that availability of decision support may have had an uneven influence on solution effectiveness and no influence on solution uniqueness. Possible implications for the design of group decision support systems for improvisation are then discussed, along with a number of observations on conducting experimentally-based research on group improvisation. 1. Introduction Emergency response organizations have benefited from technologies for information storage and retrieval (e.g., global positioning systems for locating equipment) and for communications (e.g., cellular communications networks for management of field operations). Newer information technologies and techniques are being assessed for their potential to aid in emergency response decision making. However, in order to make effective use of these technologies, reasoning systems to support human cognition are required. The exposure of these systems to unplanned-for contingencies can require personnel to develop and deploy new procedures in real- time. Creative thinking in such situations is therefore necessary in order to prevent degradation of operations, particularly when there is potential for personal injury, economic loss or environmental damage. One approach to real-time development and deployment of new procedures is improvisation. This paper discusses the design and evaluation of a computer-based system for supporting improvised decision making in simulated emergency response situations. The system’s design is informed by research on cognition in improvisation as implemented in blackboard-based computer systems. The impact of the system on group decision making in simulated emergency response scenarios is assessed by considering both subjective and objective measures of the effectiveness and uniqueness (or creativity) of the decisions made by groups. The paper proceeds as follows. Related work in supporting improvisation is first reviewed, followed by a description of the design of a system to support group improvisation in simulated emergency response situation. A series of experiments designed to evaluate the impact of availability of this system on group performance is then described, and the results of these experiments presented. The work contributes to ongoing research on improvisation by considering how group improvisation may be supported and evaluated in simulated emergency response situations. 2. Background and Related Work Emergency response relies on successful execution of one or more contingency plans, often managed by a command and control center. A commander at the scene coordinates the activities of the units fighting responding to the emergency. The on-scene commander and support staff gather and analyze data, make decisions, and monitor their implementation and consequences. The activities required to respond to an incident are often dangerous and must be performed under time pressure. The decision to activate emergency plans is based upon assessment of the potential impacts of an accident and the courses of action needed to eliminate or at least mitigate this impact. These contingency plans can rarely be executed as expected, as the case of the Exxon Valdez accident showed [1]. Flexible approaches to emergency management are therefore required. Any such approach Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03) 0-7695-1874-5/03 $17.00 © 2002 IEEE