Highly parallel particle-laden flow solver for turbulence research Peter J. Ireland, T. Vaithianathan 1 , Parvez S. Sukheswalla, Baidurja Ray, Lance R. Collins ⇑ Sibley School of Mechanical and Aerospace Engineering, Cornell University, Ithaca, NY 14853, USA International Collaboration for Turbulence Research article info Article history: Received 6 November 2012 Received in revised form 5 January 2013 Accepted 29 January 2013 Available online 16 February 2013 Keywords: Isotropic turbulence Homogeneous turbulent shear flow Inertial particles Direct numerical simulation abstract In this paper, we present a Highly Parallel Particle-laden flow Solver for Turbulence Research (HiPPSTR). HiPPSTR is designed to perform three-dimensional direct numerical simulations of homogeneous turbu- lent flows using a pseudospectral algorithm with Lagrangian tracking of inertial point and/or fluid parti- cles with one-way coupling on massively parallel architectures, and is the most general and efficient multiphase flow solver of its kind. We discuss the governing equations, parallelization strategies, time integration techniques, and interpolation methods used by HiPPSTR. By quantifying the errors in the numerical solution, we obtain optimal parameters for a given domain size and Reynolds number, and thereby achieve good parallel scaling on Oð10 4 Þ processors. Ó 2013 Elsevier Ltd. All rights reserved. 1. Introduction Turbulent flows laden with inertial particles (that is, particles denser than the carrier fluid) are ubiquitous in both industry and the environment. Natural phenomena such as atmospheric cloud formation [1–3], plankton distributions in the sea [4], and plane- tesimal formation in the early universe [5,6] are all influenced by particle–turbulence interactions. Inertial particle dynamics also impact engineered systems such as spray combustors [7], aerosol drug delivery systems [8], and powder manufacturing [9,10], among many other systems [11]. Despite extensive research, how- ever, open questions remain about the distribution of these parti- cles in the flow, their settling speed due to gravity, and their collision rates. This is due in part to our incomplete understanding of the effect of the broad spectrum of flow scales that exists at intermediate or high Reynolds numbers. Since inertial particle dynamics are strongly sensitive to the smallest scales in the flow [11], large-eddy simulation, with its associated small-scale modeling, has difficulties representing sub-filter particle dynamics accurately, including particle cluster- ing that is driven by the Kolmogorov scales [12,13]. Consequently, our investigations rely on the direct numerical simulation (DNS) of the three-dimensional Navier–Stokes equations and the Maxey and Riley equation for particle motion [14]. DNS has proven to be an extremely effective tool for investigating inertial particle dynamics, albeit at modest values of the Reynolds number due to the heavy computational demands of resolving all relevant tempo- ral and spatial scales. To extend the range of Reynolds numbers that can be simulated, we have developed a more advanced DNS code: the Highly Parallel, Particle-laden flow Solver for Turbulence Research (HiPPSTR). HiP- PSTR is capable of simulating inertial particle motion in homoge- neous turbulent flows on thousands of processors using a pseudospectral algorithm. The main objective of this paper is to document the solution strategy and numerical algorithms involved in solving the relevant governing equations. This paper is organized as follows. In Section 2, we show the equations governing the fluid and particle motion and the underly- ing assumptions of the flow solver. We discuss the time integration and interpolation techniques in Sections 3 and 4, respectively, complete with an error analysis for optimizing code performance. Section 5 then describes the parallelization strategy, and Section 6 provides conclusions. 2. Governing equations 2.1. Fluid phase The underlying flow solver is based on the algorithm presented in Ref. [15] and summarized below. It is capable of simulating both homogeneous isotropic turbulence (HIT) and homogeneous turbu- lent shear flow (HTSF) with a pseudospectral algorithm, while avoiding the troublesome remeshing step of earlier algorithms [16]. The governing equations for the flow of an incompressible 0045-7930/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.compfluid.2013.01.020 ⇑ Corresponding author at: Sibley School of Mechanical and Aerospace Engineer- ing, Cornell University, Ithaca, NY 14853, USA. Tel.: +1 607 255 9679; fax: +1 607 255 9606. E-mail address: lc246@cornell.edu (L.R. Collins). 1 Present address: INVISTA S.a.r.l., PO Box 1003, Orange, TX 77631, USA. Computers & Fluids 76 (2013) 170–177 Contents lists available at SciVerse ScienceDirect Computers & Fluids journal homepage: www.elsevier.com/locate/compfluid