Coordinate System Archive for Coevolution Wojciech Ja´ skowski and Krzysztof Krawiec, Member, IEEE Abstract—Problems in which some entities interact with each other are common in computational intelligence. This scenario, typical for co-evolving artificial-life agents, learning strategies for games, and machine learning from examples, can be formalized as test-based problem. In test-based problems, candidate solutions are evaluated on a number of test cases (agents, opponents, examples). It has been recently shown that at least some of such problems posses underlying problem structure, which can be formalized in a notion of coordinate system, which spatially arranges candidate solutions and tests in a multidimensional space. Such a coordinate system can be extracted to reveal underlying objectives of the problem, which can be then further exploited to help coevolutionary algorithm make progress. In this study, we propose a novel coevolutionary archive method, called Coordinate System Archive (COSA) that is based on these concepts. In the experimental part, we compare COSA to two state-of-the-art archive methods, IPCA and LAPCA. Using two different objective performance measures, we find out that COSA is superior to these methods on a class of artificial problems (COMPARE- ON- ONE). I. I NTRODUCTION A canonical coevolutionary algorithm evolves a population of (candidate) solutions and a population of tests based on the results of elementary interactions between them. This simple scheme is applicable to a surprisingly wide scope of test-based problems [1], including learning game strategies, machine learning from examples, artificial life, etc. Because an outcome of a single interaction is usually not informative enough, multiple outcomes are typically aggregated to drive the evaluation and selection during evolution. Unfortunately, the aggregation of outcomes is one of the reasons for which coevolutionary algorithms often suffer from so-called pathologies: over-specialization, loss of gradient, cycling [2] or disengagement [3]. These phenomena are not problematic for natural evolution that has no externally imposed goals, but make it difficult to force a coevolutionary algorithm to make a steady progress towards a specific solution concept [4] posed by the researcher. In Pareto-coevolution [5], [6] proposed to overcome some of these these drawbacks, each test gives rise to a separate objective that orders the solutions with respect to how well they fare against it, and thus the aggregation is no longer required. This transforms the test-based problem into a multiobjective optimization problem and allows relying on the well-defined concept of dominance – solution s 1 is not worse than solution s 2 if and only if s 1 performs at least as good as s 2 on all tests. Unfortunately, in real test-based problems the number of tests is usually prohibitively large; take for example the number of strategies in chess. Therefore, W. Ja´ skowski and K. Krawiec are with the Institute of Computing Science, Poznan University of Technology, Piotrowo 2, 60965 Pozna´ n, Poland; email: {wjaskowski,kkrawiec}@cs.put.poznan.pl. also the dimensionality of search space in Pareto-coevolution could be enormous. Fortunately, it was shown [7] that at least some test-based problems possess an underlying problem structure, which manifests itself by the existence of groups of tests that examine the same skill or aspect of solution performance, but with different intensity. Such tests can be ordered with respect to difficulty and placed on a common axis to form a new objective that replaces the constituent (old) objectives, leading to reduction of the dimensionality of the search space. These so-called underlying objectives [8] are typically not know a priori and have to be revealed during exploration of the problem. For instance, underlying objectives in chess could measure skills of controlling the center of the board, using knights, playing endgames, etc. The above intuition about underlying objectives and inter- nal structure of a problem was first formalized in the notion of coordinate system in [1] and then further investigated in [9], the studies which our work is based on. An important feature of coordinate system is that while compressing the initial set of objectives, it preserves the relations between solutions and tests. Each solutions is embedded in the system and the outcome of its interaction with any test can be determined given its position on all axes. The idea of extracting and using the underlying problem structure to support the progress in coevolution was first applied in Dimension Extraction Coevolutionary Algorithm (DECA) [10]. Here we propose another method, called Coordinate System Archive (COSA), that is based on similar principles. Our method differs substantially from the earlier attempt, since DECA uses a different definition of coordinate system (see Section IV) and exploits it in a different way. COSA is a proof-of-concept that the coordinate system de- fined in [1] can be also successfully used in a coevolutionary archive algorithm. The paper is organized as follows. We start with formally introducing all the necessary concepts and elaborating on the coordinate system defined by Bucci [1] in Sections II, III, and IV. After describing in details our method in Section V, we present our co-evolutionary framework in Section VI. This section describes also the LAPCA and IPCA algorithms, the problem COMPARE- ON- ONE and the experimental setup. Finally, we discuss the results in Section VII. II. MATHEMATICAL PRELIMINARIES Definition 1: Partially ordered set (poset, for short) is a pair (X, P ), where X is a set and P is a reflexive, antisymmetric, and transitive binary relation on X. We call X the ground set while P is a partial order on X. 978-1-4244-8126-2/10/$26.00 ©2010 IEEE