A Vision-Based Boundary Following Framework for Aerial Vehicles Anqi Xu and Gregory Dudek Abstract— We present an integration of classical computer vision techniques to achieve real-time autonomous steering of an unmanned aircraft along the boundary of different regions. We present two solutions, based on the same conceptual framework, to track coastlines and to follow roads surrounded by forests. In particular, we exploit color and texture properties to differentiate between the two region types in each of the aforementioned domains. The performance of these methods are evaluated using different experimental approaches, including a fully automated real-time flight outdoors over a 1km trajectory. I. INTRODUCTION In this paper we describe the design and evaluation of a system for autonomous vision-based control of an unmanned aerial vehicle (UAV). While various control systems exist for UAVs, they generally depend on global positioning system (GPS) data for short- and medium-term guidance, and are often combined with varying degrees of supervision from a human operator. In this work, we examine a control approach for fully autonomous flight based on tracking visual cues on the ground. Our work is primarily motivated by the long-term desire to follow the boundaries of environmental features such as coastlines, areas with vegetation, and large animal herds. In the short term, we aim to perform aerial reconnaissance and detect salient marine features that can be used to guide marine robotics systems. In many reconnaissance and surveillance tasks involving UAVs, a common key component is the ability to fly the ve- hicle along the boundaries of specific regions of interest. Be- cause many applications involve mapping or covering these target regions, we propose a boundary tracking framework to assist in automating these tasks. Some important applications include air-fighting forest fires, containing oil spills, and cataloging the development of geological structures. The presented work focuses on boundary tracking tasks in which the regions of interest are visually homogeneous and exhibit features that differentiate them from their surround- ings. By applying computer vision techniques to detect the location of these regions within images acquired by a UAV, we can then steer the vehicle along their boundary, in order to gather more images and repeat the process. The primary solution presented in this work focuses on tracking and following coastlines. This is part of a larger project in which aerial vehicles serve as scouts for under- water robotic systems. In addition, because the coastline tracker identifies shallow coral reefs as land, it can potentially assist in the work of biologists studying these unique and endangered marine ecosystems. The authors are with the School of Computer Science, McGill University, 3480 University Street, Montr´ eal, QC, Canada H3A 2A7 {anqixu,dudek}@cim.mcgill.ca In addition to the previous task, we believe that one of the most beneficial applications of our framework is to potentially assist firemen in suppressing forest fires. In particular, we are motivated to automatically control air- tankers and helicopters to fly and douse water along the perimeters of regions in flames, in order to contain its spread. As a proof of concept, we present a solution to detect and track highways and roads surrounded by forests. Fig. 1. The Procerus R Unicorn is a fixed-wing unmanned aerial vehicle with an on-board autopilot and gimbal-mounted camera. Powered by lithium polymer batteries, it uses radio frequency uplinks to communicate with the ground station and to stream video in real-time. This vehicle can be controlled via software or operated in semi-autonomous mode using a gamepad. The transmitted video can be processed by computer algorithms via a video capture card, and can also be displayed to human observers. We focus on the synthesis of several components of the control system to achieve fully autonomous flight for the vehicle shown in Fig. 1. We will however omit discussions on some important aspects of the problem, including vehicle dynamics, flight stabilization, and low-level image process- ing techniques involved. While each of these are instrumental to the overall performance of our system, they are based on established methods outside the scope of this paper. Our primary goals are to design a robust and real-time system. To address the speed concern, we rely predominantly on well-studied existing computer vision techniques known to be fast and reliable. To ensure robustness, we introduce fallback schemes and rejection criteria to recover from and/or filter poor intermediate results within our framework. The proposed framework locates the boundary of target regions in two stages: first, we apply cluster analysis to detect pixel regions exhibiting the desired visual features. In particular, we use color information to identify water in aerial photos of coastlines, and we use texture properties to identify roads in forest scenes. After identifying these target