Pop-Up Threat Models for Persistent Area Denial YONG LIU, Member, IEEE JOSE B. CRUZ, Jr., Life Fellow, IEEE The Ohio State University COREY J. SCHUMACHER Air Force Research Laboratory Pop-up threats usually appear or disappear randomly in a battle field. If the next pop-up threat locations could be predicted it would assist a search or attack team, such as in a persistent area denial (PAD) mission, in getting a team of unmanned air vehicles (UAVs) to the threats sooner. We present a Markov model for predicting pop-up ground threats in military operations. We first introduce a general Markov chain of order n to capture the dependence of the appearance of pop-up threats at previous locations of the pop-up threats over time. We then present an adaptive approach to estimate the stationary transition probabilities of the nth order Markov models. To choose the order of the Markov chain model for a specific application, we suggest using hypothesis tests from statistical inference on historical data of pop-up threat locations. Anticipating intelligent responses from an adversary, which might change its pop-up threat deployment strategy upon observing UAV movements, we present adaptive Markov chain models using a moving horizon approach to estimate possibly abrupt changes in transition probabilities. We combine predicted and actual pop-up target locations to develop efficient cooperative strategies for networked UAVs. A theoretical analysis and simulation results are presented to evaluate the Markov model used for predicting pop-up threats. These results demonstrate the effectiveness of cooperative strategies using the combined information of threats and predicted threats in improving overall mission performance. Manuscript received December 1, 2004; revised July 31, 2006; released for publication September 6, 2006. IEEE Log No. T-AES/43/2/903007. Refereeing of this contribution was handled by T. F. Roome. This work was supported in part by the AFRL/VA and AFOSR (Grant F33615-01-2-3154). Authors’ addresses: Y. Liu and J. B. Cruz, Jr., Dept. of Electrical and Computer Engineering, The Ohio State University, 2015 Neil Ave., Columbus, OH 43210-1272, E-mail: (yongliu6@hotmail.com, jbcruz@ieee.org); C. J. Schumacher, AFRL/VACA, 2210 8th St., Wright Patterson AFB, OH 45433-7531. 0018-9251/07/$25.00 c ° 2007 IEEE I. INTRODUCTION Ground threats in military operations are either fixed or mobile. These threats might be observed and tracked, or they might be hidden (perhaps in caves or underground) and not yet deployed. In the case of hidden threats, they become recognized as threats only after they are observed and tracked. Some threats might become hidden or taken out of deployment. Taken as a time series or sequence, these threats that appear and disappear in the course of a mission or battle are called “pop up” threats. The locations of known threats are often identified at the beginning of the battle, while a pop-up threat is not always observable prior to the start of the operation. In many cases, the probability of appearance of a pop-up threat at a location might depend on the appearance of pop-up threats at various previous locations. We capture this dependence by utilizing Markov models. The most closely related works were introduced in [1], [2], and [3]. In these references, the authors model the location of pop-up threats as a first-order Markov chain. In this case, the future appearance of a pop-up threat is dependent only on the current pop-up threat and independent of any previous pop-up threats. These papers focus on investigating the properties of the first-order Markov chain model for pop-up threats with stationary transition probabilities. In complex situations, a higher order Markov chain model might be more realistic. We introduce a higher order Markov chain model for predicting pop-up threats. We consider homogeneous (stationary) Markov chains only. Furthermore, the deployment of pop-up threats by the adversary might be changed in response to recent actions by the other force. We assume that the change of pop-up deployment could result from either a change of transition probability values or a change of order of the Markov chain process describing the deployment of the pop-up threats. The adaptive Markov chain model for pop-up threats with changing transition probabilities introduced in the work presented here is capable of changing the values of the transition probabilities and changing its order. We allow sudden changes in the transition probabilities such as step changes. A potential application associated with pop-up threats is considered here for illustration. This application is the persistent area denial (PAD) mission for multiple unmanned air vehicles (UAVs). The aim of multiple UAVs in the PAD operation is to provide persistent surveillance, tracking, or rapid engagement with strike assets, against pop-up threats at various ranges. Cooperation among multiple UAVs is a key capability for utilizing the full potential of such systems. Coordinating UAVs for a military operation has been addressed in the literature recently. In [4], the author modeled the information flow and IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 43, NO. 2 APRIL 2007 509