Genetic Programming and Evolvable Machines manuscript No. (will be inserted by the editor) A Study on Koza’s Performance Measures David F. Barrero · Bonifacio Casta˜ no · Mar´ ıa D. R-Moreno · David Camacho Received: date / Accepted: date Abstract John R. Koza defined several metrics to measure the performance of an Evolutionary Algorithm that have been widely used by the Genetic Pro- gramming community. Despite the importance of these metrics, and the doubts that they have generated in many authors, their reliability has attracted lit- tle research attention, and is still not well understood. The lack of knowledge about these metrics has likely contributed to the decline in their usage in the last years. This paper is an attempt to increase the knowledge about these measures, exploring in which circumstances they are more reliable, providing some clues to improve how they are used, and eventually making their use more justifiable. Specifically, we investigate the amount of uncertainty asso- ciated with the measures, taking an analytical and empirical approach and reaching theoretical boundaries to the error. Additionally, a new method to calculate Koza’s performance measures is presented. It is shown that these metrics, under common experimental configurations, have an unacceptable er- ror, which can be arbitrary large in certain conditions. Keywords Genetic Programming, computational effort, performance measures, experimental methods, measurement error David F. Barrero and Mar´ ıa D. R-Moreno Departamento de Autom´ atica, Universidad de Alcal´ a E-mail: david@aut.uah.es, E-mail: mdolores@aut.uah.es Bonifacio Casta˜ no Departamento de Matem´ aticas, Universidad de Alcal´ a E-mail: bonifacio.castano@uah.es David Camacho Departamento de Inform´ atica, Universidad Auton´ onoma de Madrid E-mail: david.camacho@uam.es