Handcrafted Outlier Detection Revisited Luca Cavalli 1 , Viktor Larsson 1 , Martin Ralf Oswald 1 , Torsten Sattler 2 , and Marc Pollefeys 1,3 1 Department of Computer Science, ETH Zurich, Switzerland 2 Chalmers University of Technology, Gothenburg, Sweden 3 Microsoft Mixed Reality & AI Zurich Lab Abstract. Local feature matching is a critical part of many computer vision pipelines, including among others Structure-from-Motion, SLAM, and Visual Localization. However, due to limitations in the descriptors, raw matches are often contaminated by a majority of outliers. As a result, outlier detection is a fundamental problem in computer vision and a wide range of approaches, from simple checks based on descriptor similarity to geometric verification, have been proposed over the last decades. In recent years, deep learning-based approaches to outlier detection have become popular. Unfortunately, the corresponding works rarely compare with strong classical baselines. In this paper we revisit handcrafted approaches to outlier filtering. Based on best practices, we propose a hierarchical pipeline for effective outlier detection as well as integrate novel ideas which in sum lead to an efficient and competitive approach to outlier rejection. We show that our approach, although not relying on learning, is more than competitive to both recent learned works as well as handcrafted approaches, both in terms of efficiency and effectiveness. The code is available at https://github.com/cavalli1234/AdaLAM. Keywords: low-level vision, matching, spatial matching, spatial consis- tency, spatial verification 1 Introduction Image matching is a key component in any image processing pipeline based on correspondences between images, such as Structure from Motion (SfM) [15, 41, 42, 49, 52], Simultaneous Localization and Mapping (SLAM) [3, 13, 29] and Visual Localization [8, 24, 36, 39]. Classically, the problem is tackled by computing high dimensional descriptors for keypoints which are robust to a set of transformations, then a keypoint is matched with its most similar counterpart in the other image, i.e. the nearest neighbor in descriptor space. Due to limitations in the descriptors, the set of nearest neighbor matches usually contains a great majority of outliers as many features in one image often have no corresponding feature in the other image. Consequently, outlier detection and filtering is an important problem in these applications. Several methods have been proposed for this task, from simple low-level filters based only on descriptors such as the ratio-test [27], to local spatial consistency checks [1,6,8,18,19,22,26,28,31,38,43,47,54,55,58] and global