HelipadCat: Categorised Helipad Image Dataset and Detection Method Jonas Bitoun School of Computing National University of Singapore Email: jonas@comp.nus.edu.sg Stefan Winkler School of Computing National University of Singapore Email: winkler@comp.nus.edu.sg Abstract—We present HelipadCat, a dataset of aerial images of helipads, together with a method to identify and locate such helipads from the air. Based on the FAA’s database of US airports, we create the first dataset of helipads, including a classification by visual helipad shape and features, which we make available to the research community. The dataset includes nearly 6,000 images with 12 different categories. We then train several Mask-RCNN models based on ResNet101 using our dataset. Image augmentation is applied according to learned augmentation policies. We characterize the performance of the models on HelipadCat and pick the best-performing config- uration. We further evaluate that model on the metropolitan area of Manila and show that it is able to detect helipads successfully, with their exact geographical coordinates, in another country. To reduce false positives, the bounding boxes are filtered by confidence score, size, and the presence of shadows. Dataset and code are available for download. I. I NTRODUCTION Urban Air Mobility (UAM) is expected to be integrated into the airspace [1] and expand quickly as an alternative to cars, bus or subway for inner-city or intercity commutes as well as for logistics services and deliveries. The increasing demand for these services, combined with the spread of helicopters, Unmanned Aerial Vehicles (UAVs) and autonomous drones entices the industry to make these services affordable to more people. For example, Uber has been offering trips from JFK airport to Lower Manhattan in private helicopters since July 2019. Also, startups partner with helicopter operators to offer ride-sharing platforms proposing time-saving trips by air in high-traffic areas, like Ascent Flights in the Philippines. Such co-operations allow cheaper flights and better use of available helicopters. In addition, several companies are developing their own eVTOL (electrical Vertical Take Off and Landing) aircraft [2] for short-distance urban travel as the transition to electric motors reduces the cost of vertical flights [3]. No global helipad database is available at the moment, except in the United States. The Federal Aviation Adminis- tration (FAA) lists the country’s helipad coordinates, but it is incomplete and out-of-date, as it relies on information supplied by the heliport facility owners. Automatically cataloging all heliports in a country could help overcome these issues and lead to a more complete database, for use by helicopter operators, transportation companies, and researchers. II. RELATED WORK The helipad detection problem was first addressed in [4], where a detection algorithm is designed using normalized wavelet descriptors for one specific pattern of helipad. [5] proposes a real-time detection method, with the purpose of allowing a UAV to land properly, by recognizing the in- ternational standard helipad pattern. Feature matching with Speeded-up Robust Features (SURF) [6] is used in [7] to detect one particular template of helipad in images. These algorithms are invariant to rotation and scale change, but are limited to one specific pattern. Besides, they are primarily designed to be used directly from a helicopter or an UAV for an imminent landing. They are also not database-driven. [8] applies Deep Learning to the problem, by training a CNN model based on ResNet50 [9], with Transfer Learning on the final layer, using a sliding window approach to ob- ject detection. Because this approach generates many false positives or recognizes the same helipad more than once, DBSCAN clustering algorithm [10] is applied to filter the results. The number of false positives is still too high to scale the detection to entire cities. Our contribution tackles the issue by building an accurate dataset with ground truth and categorizing the different patterns of helipads found. Based on ResNet101 and trained on more layers, we aim to improve the model’s precision. We further reduce false positive detections by applying shadow detection, since helipads are flat and free of shadows, and by filtering bounding boxes with ground areas too small or too large to fit a helipad. We make the HelipadCat dataset and code available for download. 1 III. HELIPADCAT DATASET No explicit dataset of helipad images is available. In the United States, the FAA publishes a database containing the co- ordinates of the country’s airports and helipads [11]. Based on this information, we retrieve the corresponding aerial images using the Google Maps API [12] with the respective helipad coordinates, which returns an image tile with a resolution of 640 × 640 pixels. (a) Centered (b) Small (c) Multiple Fig. 1: Examples of helipad images in the database. Out of the 5,878 images retrieved, only 3,264 (55.5%) have one or more helipads inside, for a total of 3,463 helipads. 1 https://github.com/jonasbtn/helipad detection