Z. Wahrscheinlichkeitstheorie verw. Geb. 26, 77- 86 (1973) 9 by Springer-Verlag1973 The Combinatorial Structure of Non-Homogeneous Markov Chains J. F. C. Kingman and David Williams 1. The Combinatorial Structure of Homogeneous Chains The classical theory of continuous-parameter Markov chains, as described for instance in [1], assumes that the transition probabilities P {x(t)=jIX(s)= i} (s < t), (1) where i and j run over the countable state space I, are functions pij(t-s) (2) of (t- s) alone. In this theory one important result is the Ldvy dichotomy, proved in full generality by Austin and Ornstein (see [1]), which asserts that if the func- tions p,(.) are Lebesgue measurable, then each of them is either always or never zero. Thus the relation R on I defined by R = {(i,j); p•(t)>0} (3) is independent of t > 0. A consequence of this result is that, if (i,j)eR and (j, k)~R, then for s, t>0, plk (s + t) ____ pi~ (s) pjk (t) > 0, so that (i, k) ~ R. Thus R is necessarily a transitive relation. Moreover, if the chain is standard, (i, i)eR for all i~I, so that R is reflexive. Conversely, suppose that R is any reflexive transitive relation on the countable set I. Then there exist standard chains on I which satisfy (3); consider for example a q-bounded chain whose infinitesimal generator (qij) satisfies qij>O<=~ i.j, (i,j)eR. (4) Thus the problem of characterising the relation (3) for homogeneous chains (those for which the conditional probability (1) takes the form (2)) has a very simple solution; the possible relations R are exactly the reflexive transitive relations on I. This fact has consequences for the embedding problem [3]. Thus, if (p~j; i,j e I ) is a stochastic matrix, and if there exist ~, fi, 7eI with p~>O, p~>O, p~=O, then there is no homogeneous chain whose transition probabilities satisfy pij(t)=pij, (i,jeI), for some t > O.