On the Equivalence between Set–Theoretic and Maxent MAP Estimation * Prakash Ishwar and Pierre Moulin Beckman Institute, Coordinated Science Lab, and ECE Department University of Illinois at Urbana-Champaign 405 N. Mathews Ave. Urbana, IL 61801 Contact Author: Pierre Moulin, Tel: (217) 244-8366, fax: (217) 244-8371, Email: moulin@ifp.uiuc.edu September 27, 2002 Abstract In this paper, we establish an equivalence between two conceptually different methods of sig- nal estimation under modeling uncertainty viz. set–theoretic estimation and maximum entropy (maxent) MAP estimation. The first method assumes constraints on the signal to be estimated, the second assumes constraints on a probability distribution for the signal. We provide broad conditions under which the two aforementioned estimation paradigms produce the same signal estimate. We also show how the maxent formalism can be used to provide solutions to three im- portant problems: how to select sizes of constraint sets in set–theoretic estimation (the analysis highlights the role of shrinkage); how to choose the values of parameters in regularized restora- tion when using multiple regularization functionals; and how to trade off model complexity and goodness of fit in a model selection problem. Keywords: Estimation theory, Incomplete statistics, Inverse problems, Maximum entropy methods, Modeling, Set theory, Signal restoration. EDICS Categories: 2-REST, 2-INFO, 2-SREP, 2-ESTM. ∗ This research was supported by NSF under grants MIP-97-07633 and CDA-96-24396. Parts of this work were presented at ICIP’99, Kobe, Japan; ICASSP’00, Istanbul, Turkey; and MAXENT’00, Paris, France. Prakash Ishwar is now with the EECS Department, University of California at Berkeley (ishwar@eecs.berkeley.edu). 1