Pergamon Compuring Systems in Engineering, Vol. 6, No. 3, pp. 251-259, 1995 0956-0521(95)00016-X zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPO Copyrighte 1995 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0956.0521/95 $9.50 + 0.00 PERFORMANCE OF ITERATIVE METHODS IN ANSYS ON CRAY PARALLEL/VECTOR SUPERCOMPUTERS* EUGENE L. POOLE,? MICHAEL A. HEROUX,? PRAWN VAIDYA$ and ANIL JOSHI~ tEngineering Applications Group, Cray Research, Inc., 655 Lone Oak Dr., Eagan, MN 55121, U.S.A. :Computational Applications and Systems Integration, Urbana, IL 61801, U.S.A. and Department of Computer Science, University of Illinois at Urbana-Champaign, U.S.A. Abstract-This paper describes recent work using iterative methods for the solution of linear systems in the ANSYS program. The ANSYS program, a general purpose finite element code widely used in structural analysis applications, has now added an iterative solver option. The development of robust iterative solvers and their use in commercial programs is discussed. Discussion of the applicability of iterative solvers as a general purpose solver will include the topics of robustness; as well as memory requirements and CPU performance. A new iterative solver for general purpose finite element codes which functions as a “black-box” solver using element-specific information and the underlying problem physics to construct an effective and inexpensive preconditioner is described. Some results are given from realistic examples comparing the performance of the iterative solver implemented in ANSYS with the traditional parallel/vector frontal solver used in ANSYS and a robust shifted incomplete Choleski iterative solver. INTRODUCTION With increased memory in both today’s supercom- puters (up to 4 Gwords available on the largest CRAY supercomputers) and the most powerful workstations (up to several Gbytes available) coupled with increased computational speed (Gflop perform- ance on supercomputers and Mflop performance on many workstations), users of commercial finite el- ement method (FEM) codes are now attempting to routinely solve detailed and complex simulations of physical systems. For these simulations size of prob- lems solved have grown to the point that the compu- tational cost of traditional direct methods used in many of the commercial FEM codes to solve very large linear systems is prohibitive. In addition, there is wide usage of new nonlinear solution algorithms which include the cost of solving hundreds or even thousands of linear systems. With matrix dimensions of several hundred thousand equations and wave- front sizes of several thousand, the factorization time alone becomes too large for nonlinear problems using traditional frontal solvers, even at sustained Gflop performance on the largest supercomputers. Order of magnitude reductions in computational requirements are required to solve these new grand challenge problems. Such reductions in compu- tational requirements would also allow previously defined supercomputer class problems of up to lOOk *Paper presented at the 3rd National Symposium on Large- Scale Structural Analysis for High-Performance Com- puters and Workstations, held 8-11 November 1994, Marriott Waterside, Norfolk, VA, U.S.A. degrees-of-freedom and beyond to be solved on highend workstations. Computer simulations using models containing millions of degrees-of-freedom may also become routine for the largest supercom- puters if such reductions in computational require- ments can be achieved. The development of iterative methods to solve large, sparse linear systems has been the subject of much research for the past 20 years. This research has been fueled by the dual promise of iterative methods dramatically reducing the computational require- ments for solving linear systems while effectively exploiting the use of multiple CPUs in parallel/vector and MPP (massively parallel processor) systems. This paper describes a new preconditioned conjugate gra- dient (PCG) solver which is fully implemented in the ANSYS program version 5.1, a general purpose finite element method (FEM) program. This paper will demonstrate that the new PCG solver is indeed achieving dramatic reductions in computational costs compared to the traditional frontal solvers. The new method uses an effective preconditioning strategy which derives information from the various element types used in each problem to form a very robust yet inexpensive preconditioner. Results will be presented to show the effectiveness of this solver on both a variety of standard test problems as well as a sampling of very large industrial applications prob- lems. The total solution times and computational costs for these problems are compared with the highly optimized frontal solver that has been the standard linear equation solver in the ANSYS code and most general purpose FEM codes for many years. Even 251