Navigation, localization and stabilization of formations of unmanned aerial and ground vehicles Martin Saska, Tom´ aˇ s Krajn´ ık, Vojtˇ ech Von´ asek, and Libor Pˇ reuˇ cil Abstract—A leader-follower formation driving algorithm developed for control of heterogeneous groups of unmanned micro aerial and ground vehicles stabilized under a top-view relative localization is presented in this paper. The core of the proposed method lies in a novel avoidance function, in which the entire 3D formation is represented by a convex hull projected along a desired path to be followed by the group. Such a representation of the formation provides non-collision trajectories of the robots and respects requirements of the direct visibility between the team members in environment with static as well as dynamic obstacles, which is crucial for the top-view localization. The algorithm is suited for utilization of a simple yet stable visual based navigation of the group (referred to as GeNav), which together with the on-board relative localization enables deployment of large teams of micro-scale robots in environments without any available global localization system. We formulate a novel Model Predictive Control (MPC) based concept that enables to respond to the changing environment and that provides a robust solution with team members’ failure tolerance included. The performance of the proposed method is verified by numerical and hardware experiments inspired by reconnaissance and surveillance missions. I. I NTRODUCTION Recent progress in development of autonomous micro- scale vertical take-off and landing vehicles (so called Micro Aerial Vehicles - MAVs) allows us to consider their deploy- ment in various applications, which are strictly dedicated to autonomous ground robots (UGVs) nowadays. In this paper, a scenario of multi-robot surveillance is investigated, where a formation of autonomous vehicles has to repeatedly drive through a workspace in a phalanx to cover a large operating space. MAVs can bring several advantages compared to UGVs in such a mission. For example, MAVs can reach locations inaccessible by UGVs and they may provide a top view survey of the scene, which provides an important overview for human supervisors. Nevertheless, MAVs are also handicapped by several reasons. They have low payload for sensors, they have lower operational range due to limited power source and they are difficult to control in workspaces constrained by obstacles (e.g. in abundant vegetation). These aspects make especially appealing to take advantage of both platforms and to employ a heterogeneous MAVs-UGVs team. Besides, the co-existence of ground and flying robots can provide efficient solutions of fundamental formation driving problems, as is a precise and reliable relative localization of team members closely cooperating together, which reduces probability of collisions within the robotic group. The authors are with Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Technicka 2, 166 27 Prague 6, Czech Republic {saska, vonasek, tkrajnik, preucil}@labe.felk.cvut.cz Usually, robots in reconnaissance and surveillance mis- sions may not rely on pre-installed precise global localization infrastructures and commonly available systems (as GPS) lack required precision for control of compact formations. Beside GPS lacks sufficient reliability mainly in urban and indoor environments. The proposed formation driving ap- proach is adapted for an on-board visual relative localization, which uses simple light-weight cameras mounted on MAVs and identification patterns placed on UGVs and MAVs. With this top-view concept, one may better tackle the problem of loss of direct visibility that frequently occurs in visual rela- tive localization of ground robots operating in a workspace with obstacles. The possibility of team members’ relative localization from above increases precision and reliability of the localization and brings another perspective to see the scene by operators supervising the mission. Beside the visual relative localization of individual robots, we propose to use a simple vision based technique also for the formation navigation in the environment. The presented formation driving method relies on a navigation approach called GeNav [1], which uses features detected in images gathered by a monocular camera carried by a leader of the formation. This very simple method enables to robustly navigate the group along a pre-learnt path consisting of a set of straight segments (a proof of stability of this method can be found in [1]). The combination of the top-view relative localization and visual navigation provides a light-weight, low-cost, easy- to-deploy and efficient solution, which may act as an en- abling technique for extensive utilization of simple micro- scale robots. This paper is focussed on theoretical and implementing aspects of the formation driving mechanism suited for the real-world deployment of autonomous robots under the GeNav navigation and the top-view localization, while technical details on the visual relative localization are available in [2] and the GeNav navigation in [1]. The research endeavor in the formation driving community is aimed mainly at tasks of formation stabilization [3], [4], [5], [6], [7] and/or formation following a predefined path [8], [4], [9], [10], [11]. The algorithms are designed for UGV formations [3], [8], for unmanned vehicle (UAV) formations [4], [11], for MAV (mainly quadrotors) formations [9], [10], [6], [7] or even for heterogeneous MAV-UGV formations [5]. Most of the mentioned techniques are suited for utilization of robots under a precise external global localization system (for example approaches [7], [10] are verified with the VICON system), for UGV formations they often rely on a dead- reckoning with its cumulative error [8] or they provide