Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices Maryam Fazel 1 Haitham Hindi 2 Stephen P. Boyd 3 Abstract We present a heuristic for minimizing the rank of a positive semidefinite matrix over a convex set. We use the logarithm of the determinant as a smooth approx- imation for rank, and locally minimize this function to obtain a sequence of trace minimization problems. We then present a lemma that relates the rank of any gen- eral matrix to that of a corresponding positive semidef- inite one. Using this, we readily extend the proposed heuristic to handle general matrices. We examine the vector case as a special case, where the heuristic re- duces to an iterative ℓ 1 -norm minimization technique. As practical applications of the rank minimization problem and our heuristic, we consider two examples: minimum-order system realization with time-domain constraints, and finding lowest-dimension embedding of points in a Euclidean space from noisy distance data. 1 Introduction We consider the general matrix Rank Minimization Problem (RMP) expressed as minimize Rank X subject to X ∈C , (1) where X ∈ R m×n is the optimization variable and C is a convex set, e.g., described by LMIs. It is well known that in general this problem is computationally hard to solve [VB96, §7.3]. The RMP arises in diverse areas such as control, system identification, statistics, signal processing, and computational geometry (many appli- cations are cataloged in [Faz02]). Various heuristics have been developed to handle problems of this type, specially in the context of low-order controller design; see, e.g., [BG96, SIG98, Dav94]. In this paper we describe a new heuristic for rank mini- mization that unlike the existing methods, handles any general matrix and does not require a user-specified ini- tial point. In practice, it is observed to yield low-rank 1 Contact author. California Institute of Technology, e-mail: maryam@cds.caltech.edu 2 Stanford University, e-mail: hhindi@stanford.edu 3 Stanford University, e-mail: boyd@stanford.edu Research supported in part by NSF (under ECS-9707111), and DARPA SEC (under F33615-99-C-3014). solutions, and to require only a small number of convex (semidefinite) programs to be solved. The outline of the paper is as follows. Section 2 states the semidefinite embedding lemma and its implications for the general RMP. Proofs are given in the appen- dices. Section 3 presents the log-det heuristic for the positive-semidefinite case, the general case, and the vector case. The last section discusses the applications and gives numerical examples. 2 The semidefinite embedding lemma Consider the case where the matrix variable X in the RMP (1) is constrained to be positive semidefinite (PSD). The PSD cone has properties that aid us in finding a low-rank matrix; for example, such a matrix will always lie on the boundary of the cone. In fact, this is the basis of the analytical anti-centering and potential reduction methods discussed in [Dav94]. However, there are many applications where X is not necessarily PSD, or even square, making it important to find a way to deal with the general RMP in (1). We resolve this issue by showing that it is possible to asso- ciate with any nonsquare matrix X , a positive semidef- inite matrix whose rank is exactly twice the rank of X . Thus, any general RMP can be embedded in a larger, positive semidefinite RMP. We refer to this as the semidefinite embedding lemma. Lemma 1 Let X ∈ R m×n be a given matrix. Then Rank X ≤ r if and only if there exist matrices Y = Y T ∈ R m×m and Z = Z T ∈ R n×n such that Rank Y + Rank Z ≤ 2r,  Y X X T Z  ≥ 0. (2) For the proof, see appendix A. This result means that minimizing the rank of a general nonsquare matrix X , problem (1), is equivalent to minimizing the rank of the semidefinite, block diagonal matrix diag(Y,Z ): minimize 1 2 Rank diag(Y,Z ) subject to  Y X X T Z  ≥ 0 X ∈C , (3) with variables X , Y and Z . The equivalence is in the following sense: the tuple (X ⋆ ,Y ⋆ ,Z ⋆ ) is optimal