✐ ✐ Erricos John: Handbook on Parallel Computing and Statistics DK2384 c004 2005/7/20 20:27 ✐ ✐ ✐ ✐ ✐ ✐ 4 Parallel Algorithms for the Singular Value Decomposition Michael W. Berry, Dani Mezher, Bernard Philippe, and Ahmed Sameh CONTENTS Abstract ................................................................................. 118 4.1 Introduction ........................................................................ 118 4.1.1 Basics ...................................................................... 118 4.1.2 Sensitivity of the Smallest Singular Value .................................... 120 4.1.3 Distance to Singularity — Pseudospectrum ................................... 122 4.2 Jacobi Methods for Dense Matrices ................................................. 123 4.2.1 Two-sided Jacobi Scheme [2JAC] ............................................ 123 4.2.2 One-Sided Jacobi Scheme [1JAC] ........................................... 127 4.2.3 Algorithm [QJAC] .......................................................... 130 4.2.4 Block-Jacobi Algorithms .................................................... 132 4.3 Methods for large and sparse matrices ............................................... 133 4.3.1 Sparse Storages and Linear Systems .......................................... 133 4.3.2 Subspace Iteration [SISVD] ................................................. 134 4.3.3 Lanczos Methods ........................................................... 136 4.3.3.1 The Single-Vector Lanczos Method [LASVD] ........................ 136 4.3.3.2 The Block Lanczos Method [BLSVD] ................................ 138 4.3.4 The Trace Minimization Method [TRSVD] .................................... 140 4.3.4.1 Polynomial Acceleration Techniques for [TRSVD] ................... 142 4.3.4.2 Shifting Strategy for [TRSVD] ....................................... 143 4.3.5 Refinement of Left Singular Vectors .......................................... 144 4.3.6 The Davidson Methods ...................................................... 147 4.3.6.1 General Framework of the Methods ................................. 147 4.3.6.2 How do the Davidson Methods Differ? .............................. 149 4.3.6.3 Application to the Computation of the Smallest Singular Value ....... 151 4.4 Parallel computation for Sparse Matrices ............................................ 152 4.4.1 Parallel Sparse Matrix-Vector Multiplications ................................. 152 4.4.2 A Parallel Scheme for Basis Orthogonalization ............................... 154 4.4.3 Computing the Smallest Singular Value on Several Processors ................. 156 4.5 Application: parallel computation of a pseudospectrum .............................. 157 4.5.1 Parallel Path-Following Algorithm using Triangles ............................ 157 4.5.2 Speedup and Efficiency ...................................................... 158 4.5.3 Test Problems ............................................................... 159 References .............................................................................. 160 117