arXiv:1910.10818v1 [eess.SY] 23 Oct 2019 Stochastic Reachability for Systems up to a Million Dimensions Adam J. Thorpe Electrical & Computer Eng. University of New Mexico Albuquerque, New Mexico ajthor@unm.edu Vignesh Sivaramakrishnan Electrical & Computer Eng. University of New Mexico Albuquerque, New Mexico vigsiv@unm.edu Meeko M. K. Oishi Electrical & Computer Eng. University of New Mexico Albuquerque, New Mexico oishi@unm.edu ABSTRACT We present a solution to the first-hitting time stochastic reach- ability problem for extremely high-dimensional stochastic dynamical systems. Our approach takes advantage of a non- parametric learning technique known as conditional distri- bution embeddings to model the stochastic kernel using a data-driven approach. By embedding the dynamics and un- certainty within a reproducing kernel Hilbert space, it be- comes possible to compute the safety probabilities for sto- chastic reachability problems as simple matrix operations and inner products. We employ a convergent approxima- tion technique, random Fourier features, in order to accom- modate the large sample sets needed for high-dimensional systems. This technique avoids the curse of dimensionality, and enables the computation of safety probabilities for high- dimensional systems without prior knowledge of the struc- ture of the dynamics or uncertainty. We validate this ap- proach on a double integrator system, and demonstrate its capabilities on a million-dimensional, nonlinear, non-Gauss- ian, repeated planar quadrotor system. CCS CONCEPTS • Computing methodologies → Computational control theory; Kernel methods;• Theory of computation → Sto- chastic control and optimization. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for compo- nents of this work owned by others than ACM must be honored. Abstract- ing with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. HSCC ’20, April 21-24, 2020, Sydney, Australia © 2020 Association for Computing Machinery. ACM ISBN 978-x-xxxx-xxxx-x/YY/MM. . . $15.00 KEYWORDS Stochastic Reachability, Machine Learning, Stochastic Opti- mal Control ACM Reference Format: Adam J. Thorpe, Vignesh Sivaramakrishnan, and Meeko M. K. Oishi. 2020. Stochastic Reachability for Systems up to a Million Dimen- sions. In Proceedings of Hybrid Systems: Computation and Control (HSCC ’20). ACM, New York, NY, USA, 12 pages. 1 INTRODUCTION Stochastic reachability is an established verification tech- nique which is used to compute the likelihood that a system will reach a desired state without violating a predefined set of safety constraints. The solutions to stochastic reachabil- ity problems are broadly framed in terms of a dynamic pro- gram [1, 31], which scales poorly with the dimensionality of the system. Methods using approximate dynamic program- ming [12], particle filtering [14, 17], and abstractions [28] have been posed, but are limited to systems of moderate di- mensionality. Optimization-based solutions have garnered modest computational tractability via chance constraints [14, 38], sampling methods [22, 33, 34], and convex optimization with Fourier transforms [36, 37], but are limited to linear dy- namical systems and Gaussian or log-concave disturbances. Recent work in reachability for non-stochastic, linear dy- namical systems has accommodated systems with up to a billion dimensions [4–6], an unprecendented size. However, comparably scalable solutions for stochastic systems, even with considerable structure in the dynamics and in the un- certainty, remain elusive. We propose a model-free method for stochastic reachabil- ity analysis of extremely high-dimensional systems using a class of machine learning techniques known as kernel meth- ods. Kernel methods [23] employ a data-driven approach to perform functional analysis in a high-dimensional space. Kernel methods are advantageous because they are agnos- tic to structure in the dynamics or the uncertainty, meaning that they are amenable to generic Markov control processes,