A Two-Step Evolutionary and ACO Approach for Solving the Multi-Agent Patrolling Problem Fabrice Lauri, Abderrafiâa Koukam Abstract— Patrolling an environment involves a team of agents whose goal usually consists in continuously visiting its most relevant areas as frequently as possible. For such a task, agents have to coordinate their actions in order to achieve optimal performance. Current research that tackles this complex multi-agent problem usually defines the environment as a graph, so that a wide range of applications can be dealt with, from computer network management to computer games and vehicle routing. In this paper, we consider only the instances of the multi-agent patrolling problem where all the agents are located on the same starting node. These instances are often encountered in robotics applications, where e.g. drones start from the same area, disperse over it and finally patrol around distant locations. We introduce a new Ant Colony Optimization (ACO) algorithm that is combined with an Evolutionary Algorithm (EA) technique. The novel ACO algorithm uses several ant colonies that are engaged in a competition for finding out the best multi-agent patrolling strategy. The goal of the EA is to find the best set of distant nodes enabling each agent to disperse efficiently over the graph. Experimental results show that, irrespective of the number of the involved patrolling agents and for all the graphs evaluated, our two-step EA and ACO algorithm outperforms significantly and with efficiency the best techniques proposed in the literature since now. I. I NTRODUCTION Patrolling consists in continuously visiting the relevant areas of an environment, in order to efficiently supervise, control or protect it. An ant colony searching for food and gathering it, a group of postmen on their daily rounds, or a squad of marines securing an area are all examples of a patrol. Performing such a task requires for all the involved members an efficient coordination of their actions. Most techniques that solve the multi-agent patrolling prob- lem (MAPP) use a graph as the area to be patrolled. For these reason, these techniques can be used easily in numerous applications, ranging from network management [15] to the detection of enemy threats or the protection of cities in computer games [11]. The multi-agent patrolling problem has been rigourously addressed only recently [12], [1], [16], [2], [10]. In these works, many patrolling strategies have been devised and experimentally validated using common evaluation criteria [12]. They are based on approaches of different fields, such as heuristic laws enabling agents to better choose the next node to visit [12], negotiation mechanisms [1], reinforcement learning techniques [16], schemes based on graph theory [2] or ACO [10]. Most of these solutions yield good empirical results on graphs composed of less than eighty nodes and one hundred edges. In this paper, we adopt the Ant Colony Optimization (ACO) coupled with an Evolutionary Algorithm (EA) as the solution approach to efficiently solve the multi-agent patrolling problem. Two algorithms are employed in a two-step approach for the following reasons. In some applications, especially those relating to robotics (e.g. when a group of drones have to patrol), agents are usually brought together to a given place. Under this hypothesis, the patrolling task always begins with a preliminary phase where the agents spread out over the graph. Once this phase is finished, the agents, that are now as distant from each other as possible, start patrolling from their new locations. As we intend to implement our techniques on mobile robots, we decided to only deal with situations where all the agents are located at the same node at the initial time. The most distant nodes the agents have to head for in the preliminary phase are found thanks to an Evolutionary Algorithm. Once the agents are as distant from each other as possible, begins the patrolling task. An ACO procedure is then used to determine the patrolling strategy of each agent. The novel ACO algorithm presented here uses several ant colonies that are involved in a competition for finding the best solution. Each colony tries to elaborate the best multi-agent patrolling strategy from the individual strategies found by the ants belonging to the same colony. Thus, in our ACO approach, each ant only find a partial solution to the problem. Besides, contrary to the other approaches in the literature, our ACO method directly optimize the average idleness of the graph. Using this criteria enables to yield the best patrolling strategies [3]. The applicability of ACO for the Multi-Agent Patrolling Problem (MAPP) is obvious. The problem is inherently graph-based, and clearly requires coordination and path- following behavior, tasks for which ants excel. Besides, as belonging to the family of the metaheuristics, an ACO algorithm can theoretically cope with any graph topology and any size of the agents’ population, so that a large range of situations can be considered. The remainder of this paper is organized as follows. Section II describes the commonly used framework of the patrolling problem and gives an overview of the related works. Section III presents the evolutionary algorithm that enables agents to disperse around a graph and the ACO algorithm that enables agents to patrol. Experimental results are shown in section IV and concluding remarks and future research directions are given in section V.