Noisy mean field stochastic games with network applications Hamidou Tembine LSS, CNRS-Sup´ elec-Univ. Paris Sud, France Email: tembine@ieee.org Pedro Vilanova AMCS, KAUST, Saudi Arabia E-mail:pedro.guerra@kaust.edu.sa M´ erouane Debbah Chaire Alcatel-Lucent en radio-flexible, Sup´ elec, France E-mail:merouane.debbah@supelec.fr Abstract We consider a class of stochastic games with finite number of resource states, individual states and actions per states. At each stage, a random set of players interact. The states and the actions of all the interacting players determine together the instantaneous payoffs and the transitions to the next states. We study the convergence of the stochastic game with variable set of interacting players when the total number of possible players grow without bound. We show that the optimal payoffs, the mean field equilibrium payoffs are solution of coupled system of backward-forward equations. The limiting games are equivalent to discrete time anonymous sequential games or to differential population games. Using multidimensional diffusion process, a general mean field convergence to stochastic differential equation is given. We illustrate the controlled mean field limit in wireless networks. CONTENTS I Introduction 2 II Stochastic game with individual states 4 II-A The setting ............................................. 4 II-B Policies and Strategies ...................................... 5 II-C Different types of payoffs ..................................... 5 II-D Computation of equilibria ..................................... 7 II-E Q-values .............................................. 7 II-F Dynamic team problems and stochastic potential games .................... 8 III The classic mean field approach 8 III-A Mean Field Interaction model [1] ................................ 8 III-B Mean Field Asymptotic of Markov Decision Evolutionary Games and Teams model [2] . . . 9 IV Controlled mean field interaction 10 V Noisy mean field approach 10 VI Convergence to discrete time mean field 13 VI-A Big step-size ............................................ 13 VI-A1 Mean field coordination games ............................ 14 VI-B Vanishing step-size ........................................ 15 VI-B1 Centralized mean field control ............................ 15 VII Application to malware propagation 16 VII-A Homogeneous system ....................................... 18 VII-A1 Uncontrolled behaviour ................................ 18 VII-A2 Controlled behaviour ................................. 18 VII-B Heterogeneous system ....................................... 19