Oops, Something Is Wrong Error Detection and Recovery for Advanced Human-Robot-Interaction Thorsten P. Spexard, Marc Hanheide, Shuyin Li, and Britta Wrede Abstract— A matter of course for the researchers and de- velopers of state-of-the-art technology for human-computer- or human-robot-interaction is to create not only systems that can precisely fulfill a certain task. They must provide a strong ro- bustness against internal and external errors or user-dependent application errors. Especially when creating service robots for a variety of applications or robots for accompanying humans in everyday situations sufficient error robustness is crucial for acceptance by users. But experience unveils that operating such systems under real world conditions with unexperienced users is an extremely challenging task which still is not solved satisfying. In this paper we will present an approach for handling both internal errors and application errors within an integrated system capable of performing extended HRI on different robotic platforms and in unspecified surroundings like a real world apartment. Based on the gathered experience from user studies and evaluating integrated systems in the real world, we implemented several ways to generalize and handle unexpected situations. Adding such a kind of error awareness to HRI systems in cooperation with the interaction partner avoids to get stuck in an unexpected situation or state and handle mode confusion. Instead of shouldering the enormous effort to account for all possible problems, this paper proposes a more general solution and underpins this with findings from naive user studies. This enhancement is crucial for the development of a new generation of robots as despite diligent preparations might be made, no one can predict how an interaction with a robotic system will develop and which kind of environment it has to cope with. I. INTRODUCTION Within the last decades substantial progress has been made in robotic research now enabling systems not only to perform industrial and manufacturing tasks but taking more and more part in our daily lives. This produces new challenges for robotic systems, as the more they become part of our daily life, the less the situations and scenarios are predictable in which they have to operate. Unpredictable situations, however, are difficult for robots to manage as they impose unpredictable problems and errors. In this paper we address the challenge of real-world applications imposed on robotic systems and present an approach to detect different interaction error cases and to recover and resume operation to continue HRI in a socially acceptable way. In human-human interaction the communication partners manage to maintain a common-ground by explicit communi- cation mechanisms as described in the grounding model by This work has been supported by the European Union (COGNIRON project FP6-IST-002020) and the German Research Foundation (CRC 673) The authors are with Applied Computer Science, Bielefeld Uni- versity, D-33501 Bielefeld, Germany {tspexard, mhanheid, shuyinli, bwrede}@techfak.uni-bielefeld.de Fig. 1. BARTHOC Junior demonstrating a gesture for being puzzled by scratching the back of its head Clark [1] as well as by more implicit alignment strategies as proposed by Pickering and Garrod [2]. Yet, also in human-human interaction misunderstandings arise and need to be solved by the communication partners. While in such situations it is often sufficient to clarify one propositional fact as presented in[3], it sometimes happens that a complete interaction sequence has been interpreted differently by the interlocutors. In such an extreme case one strategy is to re- start again from the beginning by explicitly “removing” the previous statements from the common ground. We present an approach based on this idea of resetting the interaction which can be triggered by both, the human user or the robot itself. While our system has been designed and evaluated on a mobile, non-humanoid robot our goal is to implement this on the humanoid robot BARTHOC (Bielefeld Anthropomorphic Robot for Human Oriented Communication) (Fig. 1) [4] in order to analyze the effects of anthropomorphism on the grounding and alignment strategies of the human user. We will first start by describing the real word conditions for our robot in Section II. In Section III we demonstrate how the proposed error model is applied to the concrete scenarios comparing alternative approaches in error detection and recovery, followed by the concrete recovery implementation for errors in Section IV we observed during multiple interac- tions. We conducted two experiments in order to evaluate the error awareness and recovery system, first testing the error awareness and recovery explicitly with varying experienced users, and secondly analyzing the interaction capabilities of the robot during an evaluation with naive users in a real flat. The results are presented in Section V. We summarize our