Diagnostic Assessment of Search Controls and Failure Modes in Many-Objective Evolutionary Optimization David Hadka dmh309@psu.edu Department of Computer Science and Engineering, The Pennsylvania State University, University Park, 16802, USA Patrick Reed pmr11@engr.psu.edu Department of Civil and Environmental Engineering, The Pennsylvania State Univer- sity, University Park, 16802, USA Abstract The growing popularity of multiobjective evolutionary algorithms (MOEAs) for solv- ing many-objective problems warrants the careful investigation of their search con- trols and failure modes. This study contributes a new diagnostic assessment frame- work for rigorously evaluating the effectiveness, reliability, efficiency and controlla- bility of MOEAs as well as identifying their search controls and failure modes. The framework is demonstrated using the recently introduced Borg MOEA, ǫ-NSGA-II, ǫ-MOEA, IBEA, OMOPSO, GDE3, MOEA/D, SPEA2 and NSGA-II on 33 instances of 18 test problems from the DTLZ, WFG and CEC 2009 test suites. The diagnostic framework exploits Sobol’s variance decomposition to provide guidance on the al- gorithms’ non-separable, multi-parameter controls when performing many-objective search. This study represents one of the most comprehensive empirical assessments of MOEAs ever completed. Keywords Evolutionary computation, multiobjective optimization, many-objective optimization, search control, parameterization. 1 Introduction Multiobjective evolutionary algorithms (MOEAs) have found favor by researchers and practitioners because of their ability to generate a Pareto approximation set in a sin- gle run for multiobjective optimization problems (MOPs). An increasingly large body of problems from diverse fields such as industrial, electrical, computer, civil and en- vironmental engineering; aeronautics; finance; chemistry; medicine; physics and com- puter science are successfully employing MOEAs (Coello Coello et al., 2007). While in the majority of these domains MOEAs have been used predominately to solve two or three objective problems, there are growing demands for addressing higher dimen- sional problems yielding a growing research community in many-objective optimization (Fleming et al., 2005; Adra and Fleming, 2009). Many-objective optimization involves the simultaneous optimization of four or more objectives. Several researchers have examined how problem difficulty is impacted by adding additional objectives to a problem. Fleming et al. (2005) studied the interac- tion between objectives. A conflicting interaction implies an improvement in one objec- tive deteriorates another objective; on the other hand, an improvement to one objective c 200X by the Massachusetts Institute of Technology Evolutionary Computation x(x): xxx-xxx