Learning Graphical Game Models Quang Duong ∗ Yevgeniy Vorobeychik † Satinder Singh ∗ Michael P. Wellman ∗ ∗ Computer Science & Engineering University of Michigan Ann Arbor, MI 48109-2121 USA † Computer & Information Science University of Pennsylvania Philadelphia, PA 19104 USA Abstract Graphical games provide compact representation of a multiagent interaction when agents’ payoffs depend only on actions of agents in their local neighborhood. We formally describe the problem of learning a graphical game model from limited observation of the payoff function, define three performance metrics for evaluating learned games, and investigate several learning algorithms based on minimizing empirical loss. Our first algorithm is a branch-and-bound search, which takes advan- tage of the structure of the empirical loss function to derive upper and lower bounds on loss at ev- ery node of the search tree. We also examine a greedy heuristic and local search algorithms. Our experiments with directed graphical games show that (i) when only a small sample of profile payoffs is available, branch-and-bound significantly out- performs other methods, and has competitive run- ning time, but (ii) when many profiles are observed, greedy is nearly optimal and considerably better than other methods, at a fraction of branch-and- bound’s running time. The results are comparable for undirected graphical games and when payoffs are sampled with noise. 1 Introduction Game theory is a formal framework for reasoning about out- comes of strategic interactions among self-interested players. This framework requires a complete specification of the play- ers’ payoff functions (payoff matrices when sets of actions are finite), which model the dependence of player utilities on the actions of the entire agent pool. To exploit common situa- tions where interactions are localized, Kearns et al. [2001] introduced graphical games, which capture dependence pat- terns as a graph structure. Graphical representations of games can induce substantial compression of the game representa- tion, and significant speedup in computing or approximating game-theoretic equilibria or other solution concepts [Duong et al., 2008]. We consider situations where there is some underlying game which can be represented compactly by a graphical model, but the payoff functions of players are unknown. We are given a data set of payoff realizations for specific strategy profiles, which are drawn according to some fixed probabil- ity distribution. For example, the game may be defined by an underlying simulator, from which we have a limited bud- get of payoff observations. The goal is to learn a graphical representation of the game based on this payoff experience. Learning graphical structures has been explored in many other contexts, notably that of inducing Bayesian network structure from data. The general version of this problem is NP-hard, and so heuristics are commonly employed, particu- larly variants of local search [Heckerman et al., 1995]. 1 In- ducing a compact representation of a joint distribution over random variables is generally formulated as an unsupervised learning problem, since we are given data reflecting the dis- tribution rather than a label specifying the distribution itself. In contrast, in our setting we induce a compact representa- tion of a payoff function given a sample of action profile and payoff tuples. Thus, like previous work on learning payoff functions given similar input [Vorobeychik et al., 2007], our problem falls in the supervised learning paradigm. A simi- lar direction was pursued by Ficici et al. [2008], who learn a cluster-based representation of games that takes advantage of game-theoretic symmetries wherever these exist. We formally define the problem of learning graphical games and introduce several techniques for solving it. One technique, a branch-and-bound search, computes an optimal solution to the empirical loss minimization problem. Other techniques are local heuristics, ranging from greedy to sev- eral variants of simulated annealing. We find that branch-and- bound can effectively take advantage of the problem structure and runs very fast (competitive with the heuristic algorithms) when the data set is not too large. On the other hand, with a large data set the greedy heuristic performs nearly as well as branch-and-bound at a fraction of running time. We provide an overview of game theory and graphical games in Section 2. Section 3 gives a formal description of the graphical game learning problem and presents algorithms for tackling it. In Section 4 we introduce three evaluation metrics for learned graphical games and present our experi- mental results. 1 Schmidt et al. [2007] present an alternative heuristic method based on L1-regularized regression. 116