1 Accepted manuscript Published in Pattern Recognition Letters 100, December 2017, Pages 96–103 c  2017. This version is made available under the CC-BY-NC-ND 4.0 license: http://creativecommons.org/licenses/by-nc-nd/4.0/ Journal version available at http://doi.org/10.1016/j.patrec.2017.10.007 Web page associated to this publication: http://gdc2016.greyc.fr/ Graph edit distance contest: Results and future challenges Zeina Abu-Aisheh a , Benoit Ga ¨ uzere b , S´ ebastien Bougleux c , Jean-Yves Ramel a , Luc Brun c , Romain Raveaux a , Pierre H´ eroux b , S´ ebastien Adam b a Universit´ e Fran¸ cois Rabelais de Tours, 64, avenue Jean Portalis, 37200 Tours, France b Normandie Univ, UNIROUEN, UNIHAVRE, INSA Rouen, LITIS, 76000 Rouen, France c Normandie Univ, UNICAEN, ENSICAEN, CNRS, GREYC UMR 6072, 6 Bd Mar´ echal Juin, 14050 Caen, France ABSTRACT Graph Distance Contest (GDC) was organized in the context of ICPR 2016. Its main challenge was to inspect and report performances and effectiveness of exact and approximate graph edit distance methods by comparison with a ground truth. This paper presents the context of this competition, the metrics and datasets used for evaluation, and the results obtained by the eight submitted methods. Results are analyzed and discussed in terms of computation time and accuracy. We also highlight the future challenges in graph edit distance regarding both future methods and evaluation metrics. The contest was supported by the Technical Committee on Graph-Based Representations in Pattern Recognition (TC-15) of the International Association of Pattern Recognition (IAPR). 1. Introduction Computing a similarity or a dissimilarity measure between graphs is a major challenge in pattern recognition. One of the most well-known and used approaches to compute a distance between two graphs is the Graph Edit Distance (GED). Com- puting the GED consists in finding a sequence of graph edit op- erations (insertions, deletions and substitutions of vertices and edges) which transforms a graph into another with a minimal cost. However, computing the GED is NP-hard. Therefore, in the last four decades, several approaches were proposed to compute approximations in polynomial time (Riesen, 2015). This paper reports the results of the Graph Distance Contest (GDC) which was organized in the context of ICPR 2016. The aim of the contest was to inspect performance and effectiveness of recent methods which compute an exact or an approximate GED. The quality of the output distances as well as the execu- tion times of the methods were used as keys for the inspection. Seven datasets were integrated, each of them being composed of several types of graphs with symbolic or numerical attributes attached to vertices and edges. GDC was open to any method which computes a sequence of edit operations transforming a graph into another one. All the participants were required to download the datasets to prepare the submission of their programs. All the programs were exe- cuted by the organizers on the same computer. Two constraints were put in the contest. First, each submitted method could not exceed 30 s per graph comparison. Second, concerning parallel methods, the number of threads was limited to 4. This paper is organized as follows: Section 2 describes the methods submitted to GDC. Then, Section 3 specifies the pro- tocol and the datasets used for this contest. Obtained results are presented and discussed in Section 4. Note that a complemen- tary and exhaustive presentation of the results is provided on GDC website http://gdc2016.greyc.fr. Last but not least, Section 5 highlights the bottlenecks of both tested methods and GED performance evaluation metrics, and proposes some pos- sible tracks to go beyond these bottlenecks. 2. Inspected methods Eight methods proposed by three different research groups were submitted. The beam search algorithm of Neuhaus et al. (2006) was also added to the list of inspected methods. All these methods can be globally divided into three categories. GED as linear or quadratic assignment problems. The GED can be reformulated as a Quadratic Assignment Problem (QAP) when the set of vertices of both graphs are extended enough by null vertices to represent removal and insertions operations (Cort´ es and Serratosa, 2013; Riesen, 2015; Bougleux et al., 2017a). Since QAP are NP-hard in general, many approxi- mation algorithms have been developed. In the GED, the no- tion of bipartite GED has been introduced in Riesen and Bunke