Theory and Methodology Comparing the ecacy of ranking methods for multiple round-robin tournaments D. Mendonc ßa * , M. Raghavachari Department of Decision Sciences and Engineering Systems, Rensselaer Polytechnic Institute, 110 8th Street, Troy, NY 12180, USA Received 1 July 1998; accepted 1 January 1999 Abstract A methodology is developed for comparing the ecacy of a number of multiple round-robin tournament ranking methods with respect to their ability to replicate the true rank order of players' strengths. Four new ranking methods are introduced in this paper. Dierent probability models for pairwise contests between tournament participants are considered. We consider the case in which there are multiple tournaments but not all players compete in all tourna- ments. Guidelines are provided for selecting a ranking method given some foreknowledge of the form of the distri- butions of the participants' strengths. Ó 2000 Elsevier Science B.V. All rights reserved. Keywords: Modeling; Simulation; Tournaments; Ranking 1. Introduction The purpose of this paper is to evaluate various methods for ranking a set of players who compete in round-robin tournaments when not all tourna- ments involve every player. Additionally, four new ranking methods for such multiple incomplete tournaments are proposed in this paper. A simu- lation methodology is developed in which two dierent models of player strength are considered. By beginning with models of player strength, the true strengths of players are therefore known, so that complete rankings of these strengths are ob- tainable. A set of round-robin tournaments is then induced and a ranking obtained via each ranking method. The four new methods and one from the tournament ranking literature are then compared for a sample problem with respect to their prop- erties and their ability to re¯ect the true ranking. The ranking methods evaluated here may be used to rank objects such as players in round-robin tournaments when not all objects may be com- pared along all criteria. Potential applications may be found in a number of areas. In evaluations of a number of related consumer products, the criteria of taste, color and smell may not be present in all the objects but it may still be desirable to form an overall ranking (Cook et al., 1996). In committee decision making, such as for project selection, not all projects may be comparable along all criteria; European Journal of Operational Research 123 (2000) 593±605 www.elsevier.com/locate/dsw * Corresponding author. Tel.: +1-518-276-2773; fax: +1-518- 276-8227. 0377-2217/00/$ - see front matter Ó 2000 Elsevier Science B.V. All rights reserved. PII: S 0 3 7 7 - 2 2 1 7 ( 9 9 ) 0 0 1 1 0 - 1