LOW-RANK MATRIX COMPLETION BY VARIATIONAL SPARSE BAYESIAN LEARNING S. Derin Babacan 1 , Martin Luessi 2 , Rafael Molina 3 , Aggelos K. Katsaggelos 2 1 Beckman Institute 2 Department of Electrical Engineering 3 Departamento de Ciencias University of Illinois and Computer Science de la Computaci ´ on e I.A. at Urbana-Champaign, USA Northwestern University, USA Universidad de Granada, Spain dbabacan@illinois.edu mluessi@northwestern.edu rms@decsai.ugr.es aggk@eecs.northwestern.edu ABSTRACT There has been a significant interest in the recovery of low-rank ma- trices from an incomplete of measurements, due to both theoretical and practical developments demonstrating the wide applicability of the problem. A number of methods have been developed for this recovery problem, however, a principled method for choosing the unknown target rank is generally missing. In this paper, we present a recovery algorithm based on sparse Bayesian learning (SBL) and automatic relevance determination principles. Starting from a ma- trix factorization formulation and enforcing the low-rank constraint in the estimates as a sparsity constraint, we develop an approach that is very effective in determining the correct rank while provid- ing high recovery performance. We provide empirical results and comparisons with current state-of-the-art methods that illustrate the potential of this approach. Index Terms— Low-rank matrix completion, Bayesian meth- ods, automatic relevance determination. 1. INTRODUCTION The problem of low-rank matrix completion (and approximation) re- cently received significant interest due to new theoretical advances [1,2] as well as interesting practical problems (e.g., the Netflix prize). Matrix completion finds application in many areas of engineering, including system identification [3], sensor networks [4], machine learning [5], computer vision [6], and medical imaging [7]. The matrix completion problem is formulated as follows. Let X ∈ R m×n be an unknown matrix with rank r ≪ min(m, n). Suppose that we only observe a subset Ω of its entries, that is, {Yij = Xij :(i, j ) ∈ Ω}. The cardinality of Ω is pmn with 0 <p ≤ 1. It has been shown in [1] that most matrices X can be recovered very accurately under certain conditions by solving the affine rank minimization problem minimize rank(X) subject to PΩ(Y)= PΩ(X), (1) where PΩ is the projection operator such that the (i, j ) th component of PΩ(X) is equal to Xij if (i, j ) ∈ Ω and zero otherwise, and Y are the observations. Since this problem is NP-hard, a popular approach is to utilize the convex relaxation based on the nuclear norm. When the observations are corrupted with noise, this problem can be stated as minimize ‖X‖∗ subject to ‖PΩ(Y) −PΩ(X) ‖ 2 F < ǫ, (2) where ‖X‖∗ is equal to the sum of the singular values of X and ‖·‖F denotes the Frobenius norm. A number of methods have been proposed for the low-rank ma- trix recovery problem. The nuclear norm based optimization prob- lem can be recast as a semidefinite program, and can be solved with interior-point solvers [3]. Singular value thresholding [8] pro- vides an attractive alternative in terms of computation. FPCA [9] introduced an efficient nuclear norm-based regularized least-squares method, whereas OPTSPACE [10] developed a method based on optimization over the Grasmann manifold with a theoretical perfor- mance guarantee for the noiseless case. A greedy approach is pro- posed in ADMIRA [11]. Finally, Bayesian methods have also been developed: a nonparametric approach for symmetric positive defi- nite matrices is proposed in [12], and a variational Bayes method is developed for collaborative filtering in [13]. Although several methods have been developed for this problem, a principled method for choosing the unknown target rank is gener- ally not motivated. In this paper, we present a recovery algorithm based on sparse Bayesian learning (SBL) principles. Based on the low-rank factorization of the unknown matrix, we employ indepen- dent sparsity priors on the individual factors with a common sparsity profile which favors low-rank solutions and simultaneously explain the observed data. Our formulation offers a few advantages over other approaches. Firstly, prior knowledge on the rank of the matrix is not required; the proposed formulation implicitly estimates the rank of the unknown matrix similarly to the automatic relevance de- termination principle in machine learning [14]. This property is not present in most of the proposed approaches (for instance, [10, 11]). Second, algorithmic parameters are treated as stochastic quantities in the proposed approach, and are handled with the combination of prior distributions and a fully-Bayesian inference procedure. In this regard, this type of formulation frees the user from extensive parameter-tuning and data- and application-dependent supervision. Finally, empirical results demonstrate that the proposed method pro- vides very good reconstruction performance compared to existing methods while accurately estimating the unknown effective rank. This paper is organized as follows. We present the proposed modeling of the problem in Section 2. Section 3 develops the estima- tion algorithm based on variational Bayesian inference. We provide empirical results in Section 4, and conclude in Section 5. 2. PROPOSED MODELING Assume that the unknown m × n matrix X is of rank r. Our model- ing is based on the following parametrization of X X = AB T , (3) 2188 978-1-4577-0539-7/11/$26.00 ©2011 IEEE ICASSP 2011