9th National Congress on Theoretical and Applied Mechanics, Brussels, 9-10-11 May 2012 Performance of a hybrid spectral/finite element solver for incompressible flows with parallel direct solver X. Dechamps 1 , G. Degrez 1 , M. Rasquin 2 1 Universit´ e Libre de Bruxelles, Department of Aero-Thermo-Mechanics CP165/41, Avenue F.D. Roosevelt 50, 1050, Brussels, Belgium 2 University of Colorado at Boulder, Department of Aerospace Engineering ECAE197, 1111 Engineering Drive, Boulder CO 80309, USA email: xdechamp@ulb.ac.be, gdegrez@ulb.ac.be, michel.rasquin@colorado.edu Abstract — The development of parallel direct solvers enabled many CFD codes to apply on finer and finer meshes. Since a decade, our department has expanded the capacities of an in-house hybrid spectral/finite element solver for 3D unsteady in- compressible flow problems with a direction of pe- riodicity. The use of a spectral development along the direction of periodicity leads to a set of 2D de- coupled systems of equations within each time step to determine the modal components of the nodal unknowns. The parallelization within the code is made both in physical and spectral domains, trying to keep as few communications between the pro- cesses as possible. We chose to couple the parallel direct solver MUMPS to our CFD code in order to solve each mode by several cores. Doing so, we can now perform computations on meshes containing tens of millions of nodes, which is now a standard in the CFD community. In the present work, we first present the nature of the parallelization made in our code. Then, on the basis of a standard valida- tion case for magnetohydrodynamic flows, we com- pare the time and memory requirements between our former direct solver and the new one. Keywords — incompressible flow, magnetohydro- dynamics, parallel direct solver, spectral/finite el- ement method I. Introduction T HE numerical simulation of turbulent magne- tohydrodynamic (MHD) flows requires the un- steady Navier-Stokes equations to be coupled with the Maxwell equations, the coupling being ensured through the Lorentz force. For such flows, the com- putations are performed on highly refined meshes and over a huge number of time steps. In modern CFD codes, one of the most challenging problems is the reduction of the solution time and memory require- ments. Direct solvers are robust methods for that purpose and can be efficient, especially if the stiff- ness matrix remains constant from time step to time step, leading to a one-time factorization of the sys- tem matrix at the first time step as presented below for our hybrid spectral/finite element solver. How- ever, their well-known restriction is their memory con- sumption. Multi-frontal approach and parallelization of these direct methods can significantly improve these two aspects without compromising their scalability for a reasonable number of processors. Their competitors, namely the iterative solvers, show a quite better mem- ory consumption and scalability for a large number of processors [1–3] but they require some preliminary tuning to have decent speed performance. Moreover, iterative solvers can hardly compete with the substitu- tion phase of direct solvers on a relatively low number of processors. The aim of this work is to couple the parallel sparse direct solver MUMPS [4–7] to our in-house CFD code SFELES [1, 3, 8–12] (which stands for Spectral/Finite Element Large Eddy Simulation) in order to take ad- vantage of the properties of our hybrid spectral/finite element approach and solve the magnetohydrodynam- ics equations on even finer meshes. II. Governing Equations T HE resolution of the MHD flow is considered in this paper, that is to say the motion of an elec- trically conducting fluid through the imposition of an external magnetic field. In a general MHD flow, the external magnetic field B 0 acts upon the veloc- ity components through the volume Lorentz force and the velocity fluctuations induce a magnetic field b. We assume here the general conditions of a labora- tory where the typical magnetic Reynolds number Re m = µ 0 σUL ≪ 1(µ 0 denotes the fluid magnetic permeability, σ its electrical conductivity, U and L are typical velocity and length scales). In such conditions, the quasi-static approximation of the MHD equations can be applied, which means that the contribution of the induced magnetic field can be neglected [13]. This approximation implies that the perturbations of the magnetic field b adjust instantaneously to flow changes and are negligible in amplitude in compari- son with that of the imposed magnetic field B 0 . 1