Generalized Expectation Criteria Andrew McCallum, Gideon Mann, Gregory Druck Department of Computer Science University of Massachusetts Amherst Amherst, MA 01003 USA {mccallum,gmann,gdruck}@cs.umass.edu DRAFT Technical Report UM-CS-2007-60 August, 2007 Abstract This note describes generalized expectation (GE) criteria, a framework for incorporating preferences about model expectations into parameter esti- mation objective functions. We discuss relations to other methods, various learning paradigms it supports, and applications that can leverage its flexibil- ity. 1 Introduction The parameters of probabilistic models are often set by maximum (aposteriori) likelihood estimation, moment matching, or the maximum entropy principle. In many common cases, these three parameter estimation methods are actually equiv- alent. However, each provides its own perspective on the parameter estimation problem; each provides different types of flexibility; and each lends itself different classes of variations outside the equivalence class. This note describes generalized expectation (GE) criteria, which in some cases also falls into the same equivalence class, and, similarly, provides yet another different perspective, a different flexibility, and useful variations outside the common equiv- alence class. Below we describe GE; we illustrate how it can be used as an aug- mentation to, or replacement for, traditional parameter estimation methods such as 1