MATHEMATICS OF COMPUTATION, VOLUME 30, NUMBER 136 OCTOBER 1976, PAGES 796-811 Factorized Variable Metric Methods for Unconstrained Optimization* By Donald Goldfarb Abstract. Several efficient methods are given for updating the Cholesky factors of a symmetric positive definite matrix when it is modified by a rank-two correction which maintains symmetry and positive definiteness. These ideas are applied to variable metric (quasi-Newton) methods to produce numerically stable algorithms. 1. Introduction. This paper is concerned with variable metric (VM) methods, (sometimes called quasi-Newton methods), for finding a local minimum of a nonlinear function, f(x), of a vector x = (xx, . . . , xn)' of n variables, where the prime "'" de- notes transposition. The fcfh iteration of a VM method, is usually expressed as (lo x(*+i> = x(fc>-ik//<fcyk>, #<*+i> = //w+/?<*>, where g^ = the gradient, V/(x), evaluated at x^, and rfc is a scalar, usually deter- mined so that either/(jc(k+1)) = min, f(x{k) - tH^g(k)) or/(jc(fc+1)) </(*<*>). H^ is a symmetric n x n matrix approximation to the inverse of the Hessian matrix, G = ffif/dXjdxA, of f(x) atx = x^k\ and E^ is a matrix, typically of rank two, which is formed from H^ and the vectors (1.2) y«) =g(k + l) _g(k) and (13) S<*)=;t(fc + 1> _*(*), subject to the condition that H<-k+'^(fc> = ps^. (p is almost always required to be 1.) The corrections E^ used in the most widely known and used VM methods (e.g., the Davidon-Fletcher-Powell (DFP), [7], [12], the complementary DFP (comp-DFP), [6], [10], [18], [25], and the rank-one [5], [8], [23], [26] methods), all belong to the one-parameter family of correction formulas 0.4) H+=H + ^^+ßrr^ s y y Hy where r = Hy/y'Hy - s/s'y [5], [18], [25]. To simplify notation, we have suppressed the superscript (k) and replaced (k + 1) by a "plus". If a line search is performed at each step, these methods can all be shown to be superlinearly convergent for f(x) strictly convex by combining Powell's elegant proof of this for the DFP method [24], with Received June 11, 1973; revised November 27, 1973, February 10, 1975, and February 17, 1976. AMS (MOS) subject classifications (1970). Primary 65F30; Secondary 15AS7, 65F0S, 65G05, 90C20, 90C30. * This research was supported in part by the National Science Foundation under Grant No. GJ 36472. Copyright © 1976, American Mathematical Society 796 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use