Shared Environment Representation for a Human-Robot Team Performing Information Fusion Tobias Kaupp, Bertrand Douillard, Fabio Ramos, Alexei Makarenko, and Ben Upcroft ARC Centre of Excellence in Autonomous Systems CAS The University of Sydney NSW 2006, Australia e-mail: t.kaupp@cas.edu.au e-mail: b.douillard@cas.edu.au e-mail: f.ramos@cas.edu.au e-mail: alexei@cas.edu.au e-mail: b.upcroft@cas.edu.au Received 13 November 2006; accepted 3 May 2007 This paper addresses the problem of building a shared environment representation by a human-robot team. Rich environment models are required in real applications for both autonomous operation of robots and to support human decision-making. Two probabi- listic models are used to describe outdoor environment features such as trees: geometric position in the world and visual. The visual representation is used to improve data as- sociation and to classify features. Both models are able to incorporate observations from robotic platforms and human operators. Physically, humans and robots form a hetero- geneous sensor network. In our experiments, the human-robot team consists of an un- manned air vehicle, a ground vehicle, and two human operators. They are deployed for an information gathering task and perform information fusion cooperatively. All aspects of the system including the fusion algorithms are fully decentralized. Experimental re- sults are presented in form of the acquired multi-attribute feature map, information ex- change patterns demonstrating human-robot information fusion, and quantitative model evaluation. Learned lessons from deploying the system in the field are also presented. © 2007 Wiley Periodicals, Inc. 1. INTRODUCTION Two problems are considered in this paper: 1 how to represent a complex environment that goes beyond geometric information, and 2 how to fuse robotic sensor information with human perceptual inputs by using the same shared environment representation in a cooperative information gathering task. • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • Journal of Field Robotics 24(11), 911–942 (2007) © 2007 Wiley Periodicals, Inc. Published online in Wiley InterScience (www.interscience.wiley.com). • DOI: 10.1002/rob.20201