Evaluation of Multi-Agent Systems: Observations from Interactive Games Rejane Frozza 1, 2 : , Yves Demazeau 1♦ , Philippe Gaudé 3 , Luis Otávio Alvares 2♥ 1 Laboratoire LEIBNIZ-IMAG – 46, Avenue Felix Viallet 38031 Grenoble, France – Yves.Demazeau@imag.fr 2 Instituto de Informática, UFRGS – Av. Bento Gonçalves, 9500 91501-970 Porto Alegre, Brazil – {frozza,alvares}@inf.ufrgs.br 3 GALILEA Multimédia, 41 bis, Rue Abbé Grégoire 38000 Grenoble, France – phil@galilea.com Abstract. Multi-agent systems can be used for a variety of applications in several domains. Interactive games are one of these applications. Multi-agent systems applied to games are a rich source of research on interaction, communication, adaptability and management of resources. This paper presents a proposal for criteria and measurements to evaluate multi-agent systems, using a game system as application. These measurements may help to analyse the operation and behaviour of multi-agent systems, helping also to the further design and development of such systems. We believe that the proposed criteria consider important features in a multi-agent environment and, therefore, can be applied to different problems represented as multi-agent systems. An example of the use of these criteria in a game prototype is described, as well as the results achieved. 1 INTRODUCTION This paper discusses the evaluation of multi-agent systems using interactive games as application. Interactive games are today a source of computational and human research. It is computational because of the technical aspects regarding system operation and development and it is human because of aspects linked to the interaction between users and the system. There are only few works about evaluation. The RoboCup research initiative [1] provides a rich environment for analysis and comparison of multi-agent systems. Some works related to this issue can be found in [2], that compares teams and attempts to analyse and generalise the lessons learned; in [3], that analyses, evaluates and understands team behaviour; and in [4], that presents ISAAC, an automated assistant to perform post-hoc analysis of RoboCup teams. More studies and proposals to compare and evaluate multi-agent systems are required. The work developed on this issue is still in an early stage, the research community needs to be able to compare its results and evaluation can be helpful from the design phase to the usage phase of a system. : CAPES fellow and Student at UFRGS (Brazil). ♦ Research Fellow at CNRS (France). ♥ Professor at UFRGS (Brazil).