Detection of Markov Chain Models from Observed Data using Binary Hypothesis Testing and Consensus ⋆ Aneesh Raghavan * John S. Baras ** * Institute for Systems Research, University of Maryland, College Park, USA (e-mail: raghava@umd.edu). ** Institute for Systems Research, University of Maryland, College Park, USA (e-mail: baras@umd.edu). Abstract: In this paper we consider the problem of detecting which Markov chain model generates observed time series data. We consider two Markov chains. The state of the Markov chain cannot be observed directly, only a function of the state can be observed. Using these observations, the aim is to find which of the two Markov chains has generated the observations. We consider two observers. Each observer observes a function of the state of the Markov chains. We formulate a binary hypothesis testing problem for each observer. Each observer makes a decision on the hypothesis based on its observations. Then Observer 1 communicates its decision to Observer 2 and vice-versa. If the decisions are the same, then a consensus has been achieved. If their decisions are different then the binary hypothesis testing problem is continued. This process is repeated until consensus has been achieved. We solve the binary hypothesis testing problem and prove the convergence of the consensus algorithm. The “value” of the information gained through 1-bit communication is discussed along with simulation results. Keywords: Hidden Markov Models, Hypothesis testing, Consensus , Value of Information. 1. INTRODUCTION Hidden Markov Models (HMM) are models in which the state of the Markov chain cannot be observed directly, instead only a function of the state can be observed. These models are used in speech recognition, econometrics, computational biology and computer vision and many other fields (Capp´ e et al. (2005)). The hypothesis testing problems have been well studied in lit- erature, one of the standard assumptions being that the observa- tions are i.i.d. Hidden Markov models are instances of models in which observations have memory and hence are not i.i.d. Chen and Willett (2000) have formulated the problem of quick- est detection of transient signals using hidden Markov models. They develop a procedure analogous to Pages test for dependent observations which can be applied to the detection of a change in hidden Markov modeled observations, i.e., a switch from one HMM to another. Lalitha et al. (2014) consider the problem where individual nodes in a network receive noisy observations whose distributions depend on the hypotheses. They analyze an update rule (for the belief of hypotheses), where each agent performs a Bayesian update based on local observations and a linear consensus among its neighbors. They prove that the belief of any agent in any incorrect hypotheses converges to zero exponentially fast. Alanyali et al. (2004) address the problem where N sensors are observing an event and obtain noisy observations. The sensor network is modeled by a graph and the sensors are restricted to exchange messages alone. They characterize conditions under which the N sensors achieve consensus and derive conditions under which the consensus converges to the centralized MAP ⋆ Research partially supported by the Army Research Office grant W911NF- 15-1-0646 to the University of Maryland, College Park. estimate. Nayyar and Teneketzis (2011) consider the problem of sequential decentralized detection where each sensor makes repeated noisy observations of a binary hypothesis. At each time the peripheral sensors need to decide whether to continue making costly observations or to send a final decision to the the fusion center. The fusion center is also faced with a stopping problem and needs to take into account the decision of the peripheral sensors. They provide parametric characterization of the optimal policies for the peripheral sensors and fusion center. In this paper, we consider two Markov chains and two ob- servers. Under the alternate hypothesis, each observer observes a different function of the state of the first Markov chain. Under the null hypothesis, each observer observes a different function of the state of the second Markov chain. Thus each observer has its own sequence of observations. Given two sequences of observations (one for each observer), the objective is to find if the sequences were generated under the null hypothesis or under the alternate hypothesis. An example of this scenario would be when there are 2 cameras observing an environment/scene and have different perspec- tives / views of the scene. The elementary events in sample space could be defined based on the environment. Consider the problem where the environment has two states . The manner in which the scene or the environment changes in each state with time is Markovian. The images (or the observations in the present example) obtained by the cameras are functions of the states of the environment. Given the images we would like to arrive at a consensus on the state of environment. For both observers , the hypothesis testing problem is for- mulated and solved as partially observed stochastic control problem. Thus both observers make individual decisions on the hypothesis. Then they communicate their decisions. If they Preprints of the 20th World Congress The International Federation of Automatic Control Toulouse, France, July 9-14, 2017 Copyright by the International Federation of Automatic Control (IFAC) 3944