processes Article Chrono::GPU: An Open-Source Simulation Package for Granular Dynamics Using the Discrete Element Method Luning Fang, Ruochun Zhang, Colin Vanden Heuvel, Radu Serban and Dan Negrut *   Citation: Fang, L.; Zhang, R.; Vanden Heuvel, C.; Serban, R.; Negrut, D. Chrono::GPU: An Open-Source Simulation Package for Granular Dynamics Using the Discrete Element Method. Processes 2021, 9, 1813. https://doi.org/10.3390/pr9101813 Academic Editor: Joanna Wi ˛ acek Received: 7 September 2021 Accepted: 8 October 2021 Published: 13 October 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affil- iations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). Department of Mechanical Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA; lfang9@wisc.edu (L.F.); rzhang294@wisc.edu (R.Z.); colin.vandenheuvel@wisc.edu (C.V.H.); serban@wisc.edu (R.S.) * Correspondence: negrut@wisc.edu Abstract: We report on an open-source, publicly available C++ software module called Chrono::GPU, which uses the Discrete Element Method (DEM) to simulate large granular systems on Graphics Processing Unit (GPU) cards. The solver supports the integration of granular material with geometries defined by triangle meshes, as well as co-simulation with the multi-physics simulation engine Chrono. Chrono::GPU adopts a smooth contact formulation and implements various common contact force models, such as the Hertzian model for normal force and the Mindlin friction force model, which takes into account the history of tangential displacement, rolling frictional torques, and cohesion. We report on the code structure and highlight its use of mixed data types for reducing the memory footprint and increasing simulation speed. We discuss several validation tests (wave propagation, rotating drum, direct shear test, crater test) that compare the simulation results against experimental data or results reported in the literature. In another benchmark test, we demonstrate linear scaling with a problem size up to the GPU memory capacity; specifically, for systems with 130 million DEM elements. The simulation infrastructure is demonstrated in conjunction with simulations of the NASA Curiosity rover, which is currently active on Mars. Keywords: granular material; Discrete Element Method; physics-based simulation; GPU computing 1. Introduction: State of the Art The Discrete Element Method (DEM) [1] is a widely-adopted scheme for predicting the dynamics of large granular systems [2], from mixing [3] and particulate flows [4] to landslides [5,6] and astrophysical processes [7]. Due to the small step size necessary for numerical stability and the large number of particles that might be required to capture the physics of interest, DEM can be computationally expensive. Until very recently, Central Processing Unit (CPU)-only parallel computing techniques have been implemented to accelerate large-scale DEM simulations. These approaches drew on (1) OpenMP for single multiprocessors with shared memory architectures [8]; (2) the Message Passing Interface (MPI) standard for clusters with distributed memory [9]; and (3) hybrid MPI–OpenMP parallelism [8,10]. An alternative architecture for parallel computing is provided by the Graphics Process- ing Unit (GPU). Over the last decade, owing to its high bandwidth and fast global memory, the GPU has anchored the intense arithmetic computations demanded, for instance, by arti- ficial intelligence applications, linear algebra, and molecular dynamics simulation. Several researchers have since established DEM codes leveraging the GPU architectures [11,12]. Nonetheless, whether using CPU or GPU computing, the number of DEM elements used in experiments reported in the literature has been rather small—in the vicinity of 10 3 to 10 5 elements [13–27]; for comparison, in one cubic meter of sand, there are on the order of two billion elements. However, one should look beyond the maximum number of elements handled to assess a DEM simulator; the element size, normal contact stiffness, and element density also come into play. In this context, the traits of a difficult DEM problem include Processes 2021, 9, 1813. https://doi.org/10.3390/pr9101813 https://www.mdpi.com/journal/processes