POMDP Autonomous Vehicle Visibility Reasoning Kyle Hollins Wray, Bernard Lange, Arec Jamgochian, Stefan J. Witwicki, Atsuhide Kobashi, Sachin Hagaribommanahalli, and David Ilstrup Alliance Innovation Lab Silicon Valley Santa Clara, CA 95051, USA Abstract—We present solutions for autonomous vehicles in limited visibility scenarios, such as traversing T-intersections, as well as detail how these scenarios can be handled simultaneously. The approach models each problem separately as a partially observable Markov decision process (POMDP). We propose an approach for integrating limited visibility within a POMDPs and implementing them on a physical robot. In order to address scala- bility challenges, we use a framework for multiple online decision- components with interacting actions (MODIA). We present the novel necessary architectural details to deploy MODIA on an actual robot. Moreover, we present a simple solution for multiple AVs to share their knowledge to overcome aspects of these limited visibility scenarios. The entire approach is demonstrated on a fully operational autonomous vehicle prototype acting in the real world at a T-intersection scenario. I. I NTRODUCTION Autonomous vehicles (AV) require the ability to reason about scenarios with limited visibility [1, 22, 6, 3]. The National Highway Traffic Safety Association determined that 94% of accidents in the U.S. are caused by human error [8]. The most prevalent cause (44% of cases) is a recognition error, in part a consequence of limited visibility. While high quality sensors (e.g., LIDARs and cameras) can greatly fa- cilitate perception of the observable environment, AVs still cannot necessarily perceive the unobservable environment behind buildings, walls, and other obstructions [22]. They are capable, however, of detecting their own limited visibility [2]. To maximize safety, AVs must reason about these “known unknowns” and intelligently make decisions for when to go, stop, or edge forward slowly for visibility when entering an occluded T-intersection or passing an obstacle in the road. This essential aspect of AV reasoning has only began to be explored recently. Approaches that introduce hand-crafted parameters [12] or a basic measure of risk [22] into the low- level motion planner’s optimization or even the go/no-go mid- level decision-making [6] have been proposed. While these more engineered approaches provide a straight-forward means to slow the AV down, they are finely tuned for very specific scenarios (e.g., 4-way uncontrolled intersections). Therefore, the approach does not necessarily scale to the wide array of scenarios found in real-world driving and does not provide any general framework or method beyond these narrow au- tonomous driving situations. The partially observable Markov decision process (POMDP) [9] provides a powerful model for sequential decision-making under limited visibility, sensor noise, and other forms of uncertainty. Specifically it can model Fig. 1. Reasoning under limited visibility about both a T-intersection (left) and passing an obstacle (right) through MODIA implemented on a fully operational autonomous vehicle prototype acting on real public roads. known sensor limitations (e.g., limited visibility) through its probabilistic model of observing other vehicles. POMDPs have only recently been embraced as a solution for general AV decision-making [1, 2, 11]. Initial attempts also used POMDPs for limited visibility in impractically large continuous [2] and discrete [5] state spaces at n-way intersections [1, 7]. Recent POMDP algorithms allow scalable AV reasoning for multiple objectives [17, 14], leverage GPUs [15], and employ nonlinear programming techniques for generalized controllers [16, 21]. These algorithms still, however, cannot scale to handle a single large monolothic POMDP. To the best of our knowledge, all prior work was done in simulation, preserving concerns about the feasibility of POMDPs, and critically lacking the details to actualize them on a robot. This paper proposes a novel solution for limited visibility reasoning in AVs using POMDPs. An exemplar limited visibil- ity scenario is considered: T-intersections. We propose a novel architectural implementation of a mathematical framework called MODIA [20, 13]. Our novel MODIA architecture fills in the critical details necessary to deploy POMDPs and MODIA on a real robot that were previously lacking. It consists of small well-defined POMDP decision problems (DPs) that are solved offline. When vehicles are perceived online, DPs are instantiated as decision components (DCs). DCs recommend an action at specific arbitration points along the route, with conflicts resolved by an executor arbitration function (e.g., take the safest action). Virtual vehicles, imagined just outside of the field-of-view, are also created and instantiate DCs to allow for reasoning about possible imperceptible vehicles. Multiple collaborative AVs are also able to share information, auto- matically affecting virtual vehicles and improving POMDP decision-making. As shown in Figure 1, this MODIA archi- tecture is successfully demonstrated on a real AV prototype. Our contributions are: (1) a general architecture for lim- ited visibility POMDP decision-making in AVs; (2) a novel POMDP solution for T-intersections; (3) a demonstration of POMDPs in MODIA on a real AV.