Adv. Appl. Prob. 48, 255–273 (2016) doi:10.1017/apr.2015.16  Applied Probability Trust 2016 PERTURBATION ANALYSIS OF INHOMOGENEOUS FINITE MARKOV CHAINS BERND HEIDERGOTT, ∗ Vrije Universiteit Amsterdam HARALAMBIE LEAHU, ∗∗ University of Amsterdam ANDREAS LÖPKER, ∗∗∗ Helmut Schmidt University GEORG PFLUG, ∗∗∗∗ University of Vienna Abstract In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t , with t> 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t , or for the integrated performance over a time interval [0,t ]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators. Keywords: Finite Markov process; inhomogeneous Markov process; sensitivity analysis; infinitesimal generator 2010 Mathematics Subject Classification: Primary 60J27 Secondary 65C05 1. Introduction Markov processes are widely used in applied probability for studying the time-dependent behavior of stochastic models. Typically, Markov processes are analyzed under the (simplify- ing) assumption that the process is homogeneous; for example, in the analysis of Markovian queueing systems it is often assumed that the arrivals do not vary over the time, whereas in reality most service systems have time-varying arrivals. There exists a variety of results on computing (approximately) the transition probability of inhomogeneous Markov processes; see, for example, [2], [13], [29], and [30]. Under appropriate smoothness conditions even the limiting behavior of inhomogeneous Markov processes can be computed; see, for example, [2] and [42]. However, the main tool for performance analysis of inhomogeneous Markov processes remains simulation. In performance analysis one is not only interested in evaluating the performance but also sensitivity analysis and optimization. In this paper we will provide a perturbation analysis for inhomogeneous Markov processes. The particular application we have in mind is that of Received 13 February 2014; revision received 20 February 2015. ∗ Postal address: Department of Econometrics and Operations Research and Tinbergen Institute, Vrije Universiteit Amsterdam, De Boelelaan 1105, 1081 HV Amsterdam, The Netherlands. Email address: b.f.heidergott@vu.nl ∗∗ Postal address: Korteweg-de-Vries Institute for Mathematics, University of Amsterdam, Science Park 107, Postbus 94248, 1090 GE Amsterdam, The Netherlands. Email address: haralambie@gmail.com ∗∗∗ Postal address: Department of Economics and Social Sciences, Helmut Schmidt University, Hamburg, 22008, Germany. Email address: lopker@hsu-hh.de ∗∗∗∗ Postal address: Department of Statistics and Operations Research, University of Vienna, Oskar-Morgenstern Platz 1, Vienna, 1090, Austria. Email address: georg.pflug@univie.ac.at 255