A Mixed-Initiative Approach to Human-Robot Interaction in Rescue Scenarios Alberto Finzi Dipartimento di Informatica e Sistemistica Universit` a degli Studi di Roma “La Sapienza” Via Salaria 113, 00198 Roma Italy finzi@dis.uniroma1.it Andrea Orlandini Dipartimento di Informatica e Automazione Universit` a degli Studi di Roma TRE Via della Vasca Navale 79, 00146 Roma Italy orlandin@dia.uniroma3.it Abstract In this paper we present a mixed-initiative planning approach to human-robot interaction in a rescue domain. We deploy a model-based executive monitoring system to coordinate the operator’s interventions and the concurrent activities of a res- cue rover. We show that this approach can enhance both oper- ator situation awareness and human-robot interaction for the execution and control of the diverse activities needed in res- cue missions. We have implemented this control architecture on a robotic system (DORO) and tested it in rescue arenas comparing its performances in different settings. Introduction Urban search and rescue (USAR) deals with response ca- pabilities for facing urban emergencies, and it involves the location and rescue of people trapped because of a struc- tural collapse. Starting in 2000, the National Institute of Standard Technology (NIST) together with the Japan Na- tional Special Project for Earthquake Disaster Mitigation in Urban Areas (Tadokoro et al. 2000; Tadokoro 2000; Maxwell et al. 2004; Jacoff, Messina, & Evans 2001) has initiated the USAR robot competitions. NIST, in particular, features future standards of robotics infrastructures, pioneer- ing robotics participation to rescue missions. RoboCup Res- cue contests are a test-bed of the technology development of NIST project, and are becoming a central international event for rescue robots, and a real challenge for the robotics community. Rescue robots uphold human operators explor- ing dangerous and hazardous environments and searching for survivors. A crucial aspect of rescue environments, discussed in (Burke et al. 2004) and (Murphy 2004) concerns the op- erator situation awareness and human-robot interaction. In (Murphy 2004) the difficulties in forming a mental model of the “robot eye” are endorsed, pointing out the role of the team. Differently from real tests, like the one in Miami (see (Burke et al. 2004)), during rescue competitions the oper- ator is forced to be alone while coordinating the robot ac- tivities, since any additional team member supporting the operator would penalize the mission. The operator can fol- low the robot activities only through the robot perception of Copyright c  2005, American Association for Artificial Intelli- gence (www.aaai.org). All rights reserved. the environment, and its internal states. In this sense, the overall control framework has to capture the operator atten- tion towards “what is important” so as to make the correct choices: follow a path, enter a covert way, turn around an unvisited corner, check whether a visible victim is really reachable, according to some specific knowledge acquired during the exploration. In this setting, a fully manual con- trol over a robot rescue is not effective (Bruemmer et al. 2003): the operator attention has to be focused over a wide range of activities, losing concentration on the real rescue mission objective, i.e. locating victims. Moreover, a sig- nificant level of training is needed to teleoperate a rescue rover. On the other hand, fully autonomous control systems are not feasible in a rescue domain where too many capabil- ities are needed. Therefore, the integration of autonomous and teleoperated activities is a central issue in rescue scenar- ios and has been widely investigated (Kiesler & Hinds 2004; Yanco & Drury 2002; Drury, Scholtz, & Yanco 2003; Michael Baker & Yanco 2004; Yanco & Drury 2002). In this work we describe a mixed-initiative planning approach (Ai-Chang et al. 2004; Myers et al. 2003; Allen & Ferguson 2002; Burstein & McDermott 1996) to Human-Robot Interaction (HRI) in a rescue domain and il- lustrate the main functionalities of a rescue robot system 1 . We deploy a model-based executive monitoring system to interface the operators’ activities and the concurrent func- tional processes in a rescue rover. In this setting, the user’s and the robot’s activities are coordinated by a continuos re- active planning process which has to (i) check the execution status with respect to a declarative model of the system; (ii) provide proactive activity while mediating among conflict- ing initiatives. In particular, we show that this approach can enhance both the operator situation awareness and human- robot interaction for the execution and control of the diverse activities needed during a complex mission such as the res- cue one. The advantage of this approach can be appreciated con- sidering the HRI awareness discussed in (Drury, Scholtz, & Yanco 2003): • robot-human interaction: given a declarative model of the robot activities, the monitoring system can be “self- aware” about the current situation, at different levels of 1 Doro is the third award winner in Lisbon contest (2004)