A maximum entropy approach to multiple classifiers combination Francois Fouss & Marco Saerens Information Systems Research Unit (ISYS) IAG – Universit´ e catholique de Louvain B-1348 Louvain-la-Neuve, Belgium Email: {saerens,fouss}@isys.ucl.ac.be March 23, 2004 Abstract In this paper, we present a maximum entropy (maxent) approach to the fusion of experts opinions, or classifiers outputs, problem. The maxent approach is quite versatile and allows us to express in a clear, rigorous, way the a priori knowledge that is available on the problem. For instance, our knowledge about the reliabil- ity of the experts and the correlations between these experts can be easily inte- grated: Each piece of knowledge is expressed in the form of a linear constraint. An iterative scaling algorithm is used in order to compute the maxent solution of the problem. The maximum entropy method seeks the joint probability den- sity of a set of random variables that has maximum entropy while satisfying the constraints. It is therefore the “most honest” characterization of our knowledge given the available facts (constraints). In the case of conflicting constraints, we propose to minimise the “lack of constraints satisfaction” or to relax some con- straints and recompute the maximum entropy solution. The maxent fusion rule is illustrated by some simulations. 1. Introduction The fusion of various sources of knowledge has been an active subject of research since more than three decades (for some review references, see [2], [5], [7]). It has recently been successfully applied to the problem of classifiers combination or fusion (see for instance [12]). Many different approaches have been developped for experts opinions fusion, including weighted average (see for instance [2], [7]), Bayesian fusion (see for in- stance [2], [7]), majority vote (see for instance [1], [11], [15]), models coming from incertainty reasoning: fuzzy logic, possibility theory [13] (see for instance [3]), stan- dard multivariate statistical analysis techniques such as correpondence analysis [17], etc. One of these approaches is based on maximum entropy modeling (see [16], [18]). Maximum entropy is a versatile modeling technique allowing to easily integrate var- ious constraints, such as correlation between experts, reliability of these experts, etc.