Preprint R AND ON ET:S HALLOW-N ETWORKS WITH R ANDOM P ROJECTIONS FOR LEARNING LINEAR AND NONLINEAR OPERATORS Gianluca Fabiani 1 , Ioannis G. Kevrekidis 2,5,6 , Constantinos Siettos 3, ∗ , Athanasios N. Yannacopoulos 4 Deep Operator Networks (DeepOnets) have revolutionized the domain of scientific machine learning for the solution of the inverse problem for dynamical systems. However, their implementation necessitates optimiz- ing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, poses a barrier to achieving high numerical accuracy. Here, inpsired by DeepONets and to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections that learn linear and nonlinear operators. The implementation of RandONets involves: (a) incorporating random bases, thus enabling the use of shallow neural networks with a single hidden layer, where the only unknowns are the output weights of the network’s weighted inner product; this reduces dramatically the dimensionality of the parameter space; and, based on this, (b) using established least-squares solvers (e.g., Tikhonov regularization and preconditioned QR decom- position) that offer superior numerical approximation properties compared to other optimization techniques used in deep-learning. In this work, we prove the universal approximation accuracy of RandONets for ap- proximating nonlinear operators and demonstrate their efficiency in approximating linear nonlinear evolution operators (right-hand-sides (RHS)) with a focus on PDEs. We show, that for this particular task, RandONets outperform, both in terms of numerical approximation accuracy and computational cost, the “vanilla” Deep- Onets. Keywords DeepOnet · RandONet · Machine Learning · Random Projections · Shallow Neural Networks · Approximation of Linear and Nonlinear Operators · Differential Equations · Evolution Operators Mathematics Subject Classification codes 65M32, 65D12, 65J22, 41A35,68T20,65D15,68T07,68W20, 41A35. 1 Introduction In recent years, significant advancements in machine learning (ML) have broadened our computational toolkit with the ability to solve both the forward and, importantly, the inverse problem in differential equations and multiscale/complex systems. For the forward problem, ML algorithms such as Gaussian process and physics-informed neural networks (PINNs) are trained to approximate the solutions of nonlinear differential equations, with a particular interest in stiff and high-dimensional systems of nonlinear differential equations [25, 47, 62, 12, 4, 14, 10, 9], as well as for the solution of nonlinear functional equations [35, 45, 34, 72, 56, 73]. The solution of the inverse problem leverages the ability of ML algorithms to learn the physical laws, their parameters and closures among scales from data [61, 62, 39, 35, 17, 13, 40, 8, 45, 34]. To the best of our knowledge, the first neural network-based solution of the inverse problem for identifying the evolution law (the right-hand-side) of parabolic Partial Differential Equations (PDEs), using spatial partial derivatives as basis functions, was presented in Gonzalez et al. (1998) [21]. In the same decade, such inverse identification problems for PDEs, were investigated through reduced order models (ROMs) for PDEs, using data-driven Proper Orthogonal Decomposition (POD) basis functions [38] and Fourier basis functions (in a context of approximate inertial manifolds) in [68]. ∗ Corresponding author at: Dipartimento di Matematica e Applicazioni “Renato Caccioppoli”, Universit` a degli Studi di Napoli “Federico II”, Naples 80126, Italy. constantinos.siettos@unina.it (1) Modelling Engineering Risk and Complexity, Scuola Superiore Meridionale, Naples 80138, Italy (2) Department of Chemical and Biomolecular Engineering, Johns Hopkins University, Baltimore 21210, MD, USA (3) Dipartimento di Matematica e Applicazioni “Renato Caccioppoli”, Universit` a degli Studi di Napoli Federico II, Naples 80126, Italy (4) Department of Statistics, Athens University of Economics and Business, Athens, Greece (5) Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore 21210, MD, USA (6) Medical School, Department of Urology, Johns Hopkins University, Baltimore 21210, MD, USA arXiv:2406.05470v1 [cs.LG] 8 Jun 2024