APPLIED STOCHASTIC MODELS AND DATA ANALYSIS Appl. Stochastic Models & Data Anal., 14, 241—253 (1998) HITTING TIME IN A FINITE NON-HOMOGENEOUS MARKOV CHAIN WITH APPLICATIONS A. PLATIS, N. LIMNIOS * AND M. LE DU Universite ´ de Technologie de Compie ´ gne, Division Mathe ´ matiques Applique ´ es B.P. 529, 60205 Compie ´ gne Cedex, France Electricite ´ de France, Direction des Etudes et Recherches 1, av. du Ge ´ ne ´ ral De Gaulle, 92141 Clamart Cedex, France SUMMARY This paper deals with the computation of the hitting time for a non-homogeneous discrete time Markov chain (NHDTMC or NHMC). We first give the basic definitions of NHMC, then we analyse the hitting time and its survivor function. We also give the sufficient conditions for the existence of the mean hitting time. Finally, a numerical example and an application of this development in reliability evaluation are given in order to illustrate our results.  1998 John Wiley & Sons, Ltd. KEY WORDS non-homogeneous Markov chain, hitting time, existence conditions of mean hitting time, reliability 1. INTRODUCTION Time of first entry, first passage time or hitting time, is defined as the random variable representing the time elapsed between the moment a system is in a state and the moment it enters a subset of states. The study of this variable is particularly important in many areas: for random walks (in physics), the hitting time represents the time a particle takes to reach a barrier, for classical gambler’s ruin problems, the hitting time stands for the duration of a game. This variable is also important in other areas of applied probability, such as reliability analysis or queuing systems. In reliability analysis, for instance, the survivor function of the hitting time is the reliability of a system and its expectation corresponds to the mean time to the first failure. In the same way, by permuting the subsets, we access other indicators such as the time a system is not repaired or the mean time to repair this system. These indicators are well known in the case of a homogeneous Markov chain modelling. Our paper deals with the case of a non-homogenous Markov chain modelling. Despite the fact that NHMC have been used in examples for manpower issues, or reliability analysis, there is not, to our knowledge, an exact theory on the existence of the mean hitting time. The objective of this paper is to give an exact computation of the hitting time and derive conditions for the existence of its mean value. To illustrate our results, we give examples and an application in reliability systems. *Correspondence to: Universite´ de Technologie de Compie´gne, Division Mathe´matiques Applique´es B.P. 529, 60205 Compie´gne Cedex, France Contract grant sponsor: Electricite´ de France; contract grant sponsor: The French Government CCC 8755 — 0024/98/030241—13 $17.50 Received October 1996  1998 John Wiley & Sons, Ltd. Revised May 1998