Appl Math Optim 38:283–301 (1998) © 1998 Springer-Verlag New York Inc. Stochastic Games with Average Payoff Criterion * M. K. Ghosh 1 and A. Bagchi 2 1 Department of Mathematics, Indian Institute of Science, Bangalore 560012, India 2 Department of Applied Mathematics, University of Twente, P.O. Box 217, 7500 AE Enschede, The Netherlands Communicated by A. Bensoussan Abstract. We study two-person stochastic games on a Polish state and compact action spaces and with average payoff criterion under a certain ergodicity condition. For the zero-sum game we establish the existence of a value and stationary optimal strategies for both players. For the nonzero-sum case the existence of Nash equilib- rium in stationary strategies is established under certain separability conditions. Key Words. Stochastic game, Stationary strategy, Value, Nash equilibrium, Ergodicity AMS Classification. 90D15, 93E05, 90D25. 1. Introduction We study noncooperative stochastic games on uncountable state and action spaces, and with ergodic or limiting average payoff. Although there is a vast literature on general state Markov decision processes (MDP) with average payoff criterion, the corresponding results on stochastic games seem to be very limited. For finite or countable state space there are several papers, e.g., [10], [22], [4], [15], [8], and [5]. Uncountable state and action spaces arise quite often in practical problems. When the planning horizon is infinite, two usual payoff criteria that are treated are discounted payoff and limiting * Part of this research was performed when the first author was visiting the University of Twente, The Netherlands.