A General Model for Online Probabilistic Plan Recognition Hung H. Bui Department of Computing Curtin University of Technology PO Box U1987, Perth, WA 6001, Australia URL: http://www.cs.curtin.edu.au/˜buihh Email: buihh@cs.curtin.edu.au Abstract We present a new general framework for online probabilistic plan recognition called the Abstract Hidden Markov Memory Model (AHM M). The new model is an extension of the existing Abstract Hidden Markov Model to allow the policy to have internal memory which can be updated in a Markov fashion. We show that the AHM M can repre- sent a richer class of probabilistic plans, and at the same time derive an efficient algorithm for plan recognition in the AHM M based on the Rao- Blackwellised Particle Filter approximate inference method. 1 Introduction The ability to perform plan recognition can be very useful in a wide range of applications such as monitoring and surveil- lance, decision supports, and team work. However the plan recognizing agent’s task is usually complicated by the uncer- tainty in the plan refinement process, in the outcomes of ac- tions, and in the agent’s observations of the plan. Dealing with these issues in plan recognition is a challenging task, es- pecially when the recognition has to be done online so that the observer can react to the actor’s plan in real-time. The uncertainty problem has been addressed by the sem- inal work [Charniak and Goldman, 1993] which phrases the plan recognition problem as the inference problem in a Bayesian network representing the process of executing the actor’s plan. More recent work has considered dy- namic models for performing plan recognition online [Py- nadath and Wellman, 1995; 2000; Goldmand et al., 1999; Huber et al., 1994; Albrecht et al., 1998]. While this offers a coherent way of modelling and dealing with various sources of uncertainty in the plan execution model, the computational complexity and scalability of inference is the main issue, es- pecially for dynamic models. Inference in dynamic models such as the Dynamic Bayesian Networks (DBN) [Nicholson and Brady, 1994] is more difficult than in a static model. Inference in a static network utilizes the sparse structure of the graphical model to make it tractable. In the dynamic case, the DBN belief state that we need to maintain usually does not preserve the conditional independence properties of the single time-slice network, making exact inference intractable even when the DBN has a sparse structure. Thus, online plan recognition algorithms based on exact inference will run into problems when the belief state becomes too large, and will be unable to scale up to larger or more detailed plan hierarchies. In our previous work, we have proposed a framework for online probabilistic plan recognition based on the Abstract Hidden Markov Models (AHMM) [Bui et al., 2002]. The AHMM is a stochastic model for representing the execution of a hierarchy of contingent plans (termed policies). Scal- ability in policy recognition in the AHMM is achieved by using an approximate inference scheme known as the Rao- Blackwellised Particle Filter (RBPF) [Doucet et al., 2000]. It has been shown that this algorithm scales well w.r.t. the number of levels in the plan hierarchy. Despite its computational attractiveness, the current AHMM is limited in its expressiveness, in particular, its in- ability to represent an uninterrupted sequence of plans and actions. This is due to the fact that each policy in the AHMM is purely reactive on the current state and has no memory. This type of memoryless policies cannot represent an unin- terrupted sequence of sub-plans since they have no way of remembering the sub-plan in the sequence that is currently being executed. In other words, the decision to choose the next sub-plan can only be dependent on the current state, and not on the sub-plans that have been chosen in the past. Other models for plan recognition such that the Probabilistic State Dependent Grammar (PSDG) [Pynadath and Wellman, 2000; Pynadath, 1999] are more expressive and do not have this limitation. Unfortunately, the existing exact inference method for the PSDG in [Pynadath, 1999] has been found to be flawed and inadequate [Bui, 2002]. The main motivation in this paper is to extend the existing AHMM framework to allow for policies with memories to be considered. We propose an extension of the AHMM called the Abstract Hidden Markov mEmory Model (AHM M). The expressiveness of the new model encompasses that of the PSDG [Pynadath and Wellman, 2000], thus the new model removes the current restriction of the AHMM. More impor- tantly, we show that the RBPF approximate inference method used for the AHMM can be extended to the more general AHM M as well, ensuring that the new generalized model remains computationally attractive. To the best of our knowl- edge, we are the first to provide a scalable inference method