Uncertainty Measured Markov Decision Process in Dynamic Environments Sourav Dutta 1 , Banafsheh Rekabdar 2 , and Chinwe Ekenna 1 Abstract Successful robot path planning is challenging in the presence of visual occlusions and moving targets. Classical methods to solve this problem have used visioning and perception algorithms in addition to partially observable markov decision processes to aid in path planning for pursuit- evasion and robot tracking. We present a predictive path planning process that mea- sures and utilizes the uncertainty present during robot motion planning. We develop a variant of subjective logic in combi- nation with the Markov decision process (MDP) and provide a measure for belief, disbelief, and uncertainty in relation to feasible trajectories being generated. We then model the MDP to identify the best path planning method from a list of possible choices. Our results show a high percentage accuracy based on the closest acquired proximity between a target and a tracking robot and a simplified pursuer trajectory in comparison with related work. I. I NTRODUCTION Planning paths for robots in narrow and uncertain spaces e.g, boulders falling causing blocked areas is still a difficult task. In all types of environments that a robot may encounter (if they’re scalable), they are confronted with various kinds of decisions involving multiple choices and relative uncertainty. A clear understanding of these uncertainties becomes a prerequisite for effective decision making. Recently an innovative logic variant method was devel- oped [5], called Collective Subjective Logic (CSL), and the authors created models for minimizing uncertainty in a deci- sion making process by using Probabilistic Soft Logic (PSL). They combined multiple opinions concurrently and provided high scalability and prediction accuracy while dealing with uncertain opinions. This work will take insight from the aforementioned work, make innovative advancements and apply them to path planning problems. Interestingly, some important work in robotics that looks into belief states and partially observable processes have been developed in [4], [10], [17]. The models, however, see an exponential growth in their running time due to the increasing number of states needed as more unknown locations are explored. Improved variants of these models have been developed but, the measurement of uncertainty present and how it can mitigate longer planning times has not been investigated. This research is supported in part by NSF awards CRII-IIS-1850319 1 Sourav Dutta and Chinwe Ekenna - Department of Computer Sci- ence, University at Albany, SUNY, NY 12206, USA {sdutta2, cekenna}@albany.edu. 2 Banafsheh Rekabdar - Department of Computer Sci- ence Southern Illinois University, IL 62901, U.S.A. banafsheh.rekabdar@siu.edu. Fig. 1: Process Overview In this paper, we introduce a novel combination of the CSL and MDP which we call Uncertainty- MDP (U-MDP). Our method has two main modules as illustrated in Figure 1 : the learning phase; where the algorithm uses a combination of PSL and Subjective Logic (SL) to learn the uncertainty representations of the available motion planning algorithms and the prediction phase; where the MDP uses the evidence gathered in the learning phase to predict the planning algo- rithm in use. Compared to other MDP variants, the states in our stochas- tic process don’t increase exponentially instead we utilise available planning strategies to induce on the states of the MDP. We make measurements of the calculated uncertainty in addition to belief and disbelief, to pick the best choice of planning strategy in a given environment. Using our approach thus limits the search space and reduces computational complexity. The contributions of this paper include: • an improved target tracking algorithm that takes obser- vational and estimation uncertainties into account and uses them in the decision making process and • a faster algorithm with reduced execution time with a reduction of an exponentially increasing belief space to a fixed number dependent on the number of planning strategies available. II. RELATED WORK In this section, we discuss work related to logic systems, Markov decision process, and application to robotics.