Using Fitness as Minimal Criteria for Novelty Search in the Maze Navigation Domain Paulo Urbano 1 Henrique Vaz 2 Abstract. The problem of deception is one of the biggest chal- lenges for evolutionary robotics. Some fitness functions suffer from deception because they prevent the objective from being reached misguiding the search process towards local optima and poor solu- tions. A lot of techniques were introduced to overcome deception, mainly through the promotion of genotypic diversity. Recently, a radically different and counter-intuitive open-ended evolutionary ap- proach called Novelty Search, which rewards behavioral phenotype novelty, was proposed with very successful results. The main idea is, paradoxically, to find what we want without explicitly looking for it. But as novelty search is guided by behavior novelty alone, its reliability can be greatly affected when searching through vast and unconstrained behavior spaces: it may spend most of its time ex- ploring irrelevant behaviors with respect to the goal. To address this problem, different techniques were proposed, mostly of them rein- troducing the fitness function. One of them is Progressive Minimal Criteria Novelty Search (PMCNS), which demonstrated promising results on a swarm robotics task. In PMCNS, novelty search freely explores new regions of behavior space as long as the solutions meet a progressively stricter fitness criterion. We have empirically eval- uated PMCNS in a maze navigation simulated task, using a decep- tive maze and an unclosed variation of it, representing respectively very deceptive problems with and without a vast behavior search space. In both mazes a simulated robot controlled by an evolved neural network, using NEAT, had to find a goal zone. The perfor- mance of PMCNS was compared with two other methods that were designed to prune the space of relevant behaviors: Minimal Criteria Novelty Search (MCNS) and Linear Scalarization of novelty and fit- ness (LS). MCNS was tested with domain dependent minimal criteria and also with a fixed fitness threshold in order to see if the progres- sion and adaptation of the minimal criteria was necessary. We have also tested the performance of pure Novelty Search and the standard fitness based search. The experiments results showed that the use of a fitness threshold as the minimal criteria for constraining novelty search is a promising technique, specially in the unclosed deceptive maze with a large behavior space. 1 INTRODUCTION Deception is one of the biggest challenges in evolutionary robotics (ER). Because of deception, some fitness functions can misguide the search process towards local optima, preventing the objective from being reached and thus resulting in poor solutions. The more com- 1 LabMAg-Faculty of Sciences, University of Lisboa, Portugal, E-mail: pub@di.fc.ul.pt 2 LabMAg-Faculty of Sciences, University of Lisboa, Portugal, E-mail: hen- rique.vaz.91@gmail.com plex the goal task is, the harder it may be to define a non-deceptive fitness function that demands more domain knowledge and careful shaping. The interactions between a robot and its environment are often complex, even in simple tasks and fitness functions in ER are therefore prone to be deceptive. Diversity promotion and maintenance methods (see [14], [7]), one of the main approaches towards overcoming deception, are usually performed in the genotypic space. Recently diversity techniques per- formed on the phenotypic behavior space (see [3], [9], [16], [6], [23]) have achieved very good results, mainly in the area of neuroevolu- tionary robotics, where there is an evaluation in a noisy environment and there are complex discontinuous mappings between genotype and phenotype behavior. Big differences in genotype may generate the same behavior and small differences in the genotype can radi- cally change the resulting behaviors. ”For some problem classes and representations, applying a particular metric in one space is equiva- lent to applying it in the other: the genotype-phenotype mapping, G, preserves the relative distance between objects in each. When this is not the case, it may be more informative to measure the similarity between phenotypes (...), after all, what we are truly interested in is the similarity of solutions, not their encodings.” [6]. The promotion of diversity behavior has been used in the con- text of standard objective based search (see [4], [17], [24]) but re- cently, Lehman and Stanley [9] proposed a radically different evolu- tionary approach called novelty search (NS). Novelty search simply searches for novel behaviors regardless of their fitness quality, over- coming deception by ignoring the objective. In NS, behaviors are scored based on how different they are from previously evaluated behaviors. The approach has been successfully applied to many dif- ferent domains, including evolutionary robotics (see [4], [8], [12], [19]). Besides avoiding getting stuck in local optima, it has been demonstrated that NS is able to find more diverse and less complex solutions, when compared to objective-based evolution [9]. As NS is guided by behavioral innovation alone, its performance can be greatly affected when searching through vast behavior spaces (see [1], [11]) it may spend most of its time exploring behaviors that are not relevant for the goal task, scoring low fitness scores. To address this problem, Lehman and Stanley [11] proposed mini- mal criteria novelty search (MCNS). MCNS is an extension of NS where individuals must meet some domain-dependent minimal cri- teria to be selected for reproduction. In [11], the authors applied MCNS in two maze navigation tasks and demonstrated that MCNS evolved solutions more consistently than both novelty and fitness- based search. However, MCNS suffers from three major drawbacks: domain knowledge is required to define suitable minimal criteria; it may be necessary to bootstrap the search with a genome specifically evolved to satisfy the criteria; and we have to take into account the