Elementary subpaths in discounted stochastic games Kimmo Berg Department of Mathematics and Systems Analysis, Aalto University School of Science, P.O. Box 11100, FI-00076 Aalto, Finland Abstract This paper examines the subgame-perfect equilibria in discounted stochastic games with finite state and action spaces. The fixed-point characterization of pure-strategy equilibria is generalized to unobservable mixed strategies. It is also shown that the pure-strategy equilibria consist of elementary subpaths, which are repeating fragments that give the acceptable action plans in the game. The developed methodology offers a novel way of computing and analyzing equilibrium strategies that need not be stationary nor Markovian. Keywords: game theory, stochastic game, subgame-perfect equilibrium, equilibrium path, fixed-point equation, tree 1. Introduction Stochastic games are multiplayer decision making models in a dynamic en- vironment. They have been introduced in a seminal paper by Shapley (1953) and they can be applied, e.g., in industrial organization (Ericson and Pakes 1995; Pakes and Ericson 1998), taxation (Phelan and Stacchetti 2001), fish wars, stochastic growth models, communication networks, queues, and hiding and searching for army forces; see the references in Filar and Vrieze (1997); Amir (2003); Doraszelski and Pakes (2007); Balbus et al. (2014). The stochastic game model extends both Markov decision processes (MDPs) which have only a sin- gle decision maker and repeated games where the players encounter the same game over and over again. In contrast to MDPs, the stochastic games may have multiple solutions and a set of possible equilibrium payoffs that depends on the players’ strategies. Moreover, the strategies are inherently more complicated compared to repeated games as the actions can be planned conditionally based on the realized future state. This paper examines discounted stochastic games with a finite number of players, actions and states. The existence of an equilibrium in stationary strate- gies has been proven in Fink (1964) and Takahashi (1964). For more general stochastic games, the existence of an equilibrium is still an open problem. For Email address: kimmo.berg@aalto.fi (Kimmo Berg) Preprint submitted to - April 7, 2015