Collaborative Object Picking and Delivery with a Team of Micro Aerial Vehicles at MBZIRC Matthias Nieuwenhuisen, Marius Beul, Radu Alexandru Rosu, Jan Quenzel, Dmytro Pavlichenko, Sebastian Houben, and Sven Behnke Abstract— Picking and transporting objects in an outdoor environment with multiple lightweight MAVs is a demanding task. The main challenges are sudden changes of flight dynamics due to altered center of mass and weight, varying lighting conditions for visual perception, and coordination of the MAVs over unreliable wireless connections. At the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) teams competed in a Treasure Hunt where three MAVs had to collaboratively pick colored disks and drop them into a designated box. Only little preparation and test time on- site required robust algorithms and easily maintainable systems to successfully achieve the challenge objectives. We describe our multi-robot system employed at MBZIRC, including a lightweight gripper, a vision system robust against illumination and color changes, and a control architecture allowing to operate multiple robots safely. With our system, we—as part of the larger team NimbRo of ground and flying robots—won the Grand Challenge and achieved a third place in the Treasure Hunt. I. I NTRODUCTION Aerial manipulation—especially picking, transporting, and delivering objects—became an area of much interest in recent years. Micro aerial vehicles (MAV) are well suited to quickly deliver small, but valuable objects, e.g., spare parts or medical substances. A particular advantage of employing aerial vehicles to detect and pick objects is that—in contrast to ground vehicles—they can reach otherwise hard to access or even dangerous areas. To facilitate the development in this field, one of the tasks at the Mohamed Bin Zayed International Robotics Challenge 2017 (MBZIRC) was collaborative picking with MAVs. The Treasure Hunt task was to find and pick colored, ferromagnetic discs from the ground of an outdoor arena and to deliver them to a designated box in a predefined drop zone. Fig. 1 shows one of our MAVs while picking a moving object. Teams were provided with rough specifications of the ob- jects, i.e., diameter, height above ground, maximum weight of 500 g, and the possible colors, in advance. The drop box was specified by its approximate dimensions. Nevertheless, the exact arena setup—including colored markings on the ground making color-based perception challenging—was not known in advance and teams had to develop robust and flexible systems. This work has been supported by a grant of the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) and grants BE 2556/7-2 and BE 2556/8-2 of the German Research Foundation (DFG). The authors are with the Autonomous Intelligent Systems Group, Computer Science VI, University of Bonn, Germany nieuwenh@ais.uni-bonn.de Fig. 1. Picking a dynamic object. Our MAV follows the yellow disc with visual servoing. The telescopic rod and the ball joint of our electromagnetic gripper allow compliant picking without disturbing attitude control of the MAV. The picked objects were delivered to a drop box up to 75 m away. In contrast to lab experiments or controlled field tests, the particular challenge of the competition was the much reduced testing time. The arena was only accessible for teams at assigned time slots. In total, individual teams had two rehearsal slots of 35 minutes and four competition trials— two for the Treasure Hunt and two for the Grand Challenge. The systems had to be set up in the arena in only five minutes in each run. Consequently, complex algorithms that need extensive fine-tuning or are prone to failing in some cases are not an option for a competition system. Thus, we focused on simple but robust approaches and tried to identify and cover as many issues in advance as possible. Experiences gained during trials had to be incorporated into the system without additional testing before the next trial. Hence, the system complexity needed to be as low as possible to eliminate error sources. Furthermore, quickly changing overcast and light sandstorms changed illumination and navigation conditions significantly from trial to trial. Due to these challenging conditions, only very few teams managed to score in this task autonomously. Our main contributions are • robust detection of objects with only roughly specified color under varying lighting conditions, • relative navigation while picking and dropping, • lightweight and flexible picking hardware, • coordination of multiple MAV with and without WiFi connections, and European Conference on Mobile Robotics (ECMR), Paris, France, September 2017.