© J.C. Baltzer AG, Science Publishers On performance potentials and conditional Monte Carlo for gradient estimation for Markov chains Xi-Ren Cao a , Michael C. Fu b and Jian-Qiang Hu c a Department of Electrical Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong b The Robert H. Smith School of Business, Institute for Systems Research, University of Maryland, College Park, MD 20742, USA c Department of Manufacturing Engineering, Boston University, Boston, MA 02215, USA We consider the problem of sample path-based gradient estimation for long-run (steady- state) performance measures defined on discrete-time Markov chains. We show how two estimators – one derived using the likelihood ratio method with conditional Monte Carlo and splitting, and the other derived using performance potentials and perturbation analysis – are related. In particular, one can be expressed as the conditional expectation of a suitably weighted average of the other. This demonstrates yet another connection between the two gradient estimation techniques of perturbation analysis and the likelihood ratio method. Keywords: Markov chains, perturbation analysis, likelihood ratio method, gradient esti- mation, potential theory 1. Introduction Many stochastic systems can be modeled by Markov chains with a finite state space. However, due to the complexity of realistic systems of interest, the resulting model may have an enormous state space that leads to intractability in terms of com- putational complexity. Large closed queueing networks are a classic example of this phenomenon. In this case, it is often more computationally efficient to simulate the system rather than to try to solve it analytically. In order to perform sensitivity analysis or optimization when simulation is employed, various gradient estimation techniques have been developed over the past two decades, most notably the techniques of pertur- bation analysis (cf. Ho and Cao [14], Glasserman [13], Cao [1] and Fu and Hu [11]) and the likelihood ratio method, which is also sometimes referred to as the score function method (cf. Rubinstein and Shapiro [15]); see also Fu [10] for a review of Annals of Operations Research 87(1999)263–272 263