arXiv:2104.04636v1 [math.PR] 9 Apr 2021 C ONTINUOUS -T IME HIGHER ORDER MARKOV C HAINS :F ORMULATION AND PARAMETER E STIMATION S URYADEEPTO NAG * Abstract Stochastic processes find applications in modelling systems in a variety of disciplines. A large number of stochastic models considered are Markovian in nature. It is often observed that higher order Markov processes can model the data better. However most higher order Markov models are discrete. Here, we propose a novel continuous-time formulation of higher order Markov processes, as stochastic differential equations, and propose a method of parameter estimation by maximum likelihood methods. 1 I NTRODUCTION While Markov chains have been used in a variety of disciplines such as biology and economics [1], the usage of higher order Markov chains is relatively sparse. This may be attributed to the significantly larger number of parameters one encounters in higher order Markov models [2]. However, the Markovian nature of many models often fail to stand the test of data. In [3], Marathe and Ryan examined the validity of modelling sets of data as Markovian geometric brownian motions, and they observed instances where there exists a dependence of incre- ments in the state variable on the past increments. In [4], Shorrocks investigated the Markovian assumption in modelling income mobility and concluded that a second order model would be more appropriate. Another disadvantage of using a higher order Markov chain in modelling so far, has been in the discrete na- ture of the models. We observe that most instances of higher order Markov models which have been used so far, have involved discrete time models. This leaves out a very large class of stochastic processes which cannot be modelled using higher order Markov models as they are inherently continuous in nature. Discretizing such systems not only diminishes the accuracy of modelling, but also significantly increases the “order” of the now discrete model, in turn increasing the number of parameters in the model. Despite these factors, higher order Markov chains have elicited sustained interest [5]. The objective of this paper is to develop a continuous-time formulation of higher order Markov chains. The inclusion of continuous processes under higher order Markov models will make it possible to model better, sev- eral systems, which were till now modelled either by Markov processes or by discrete-time higher order Markov processes. Under our formulation, continuous-time higher order Markov processes are modelled with the order being an interval of time instead of a number of discrete states. The state variable is written as a function of an underlying time variable, such that the “state function” represents the state variable over all instances of time within the interval with length equal to the order. Drift and diffusion functions are also written in terms of the underlying variable. This allows higher order Markov processes to be modelled as stochastic differential equa- tions, as is done with Markov processes. We subsequently derive the Fokker-Planck equation in terms of these * Indian Institute of Science Education and Research Pune, Pune-411008, Maharashtra, India, e-mail: suryadeepto.nag@students.iiserpune.ac.in 1