arXiv:1512.07669v1 [math.OC] 23 Dec 2015 Reinforcement Learning Stochastic Approximation Algorithms for Markov Decision Processes Vikram Krishnamurthy University of British Columbia, Vancouver, Canada. V6T 1Z4. vikramk@ece.ubc.ca December 2015