4118 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 60, NO. 8, AUGUST 2012 Sparsity-Exploiting Robust Multidimensional Scaling Pedro A. Forero, Student Member, IEEE, and Georgios B. Giannakis, Fellow, IEEE Abstract—Multidimensional scaling (MDS) seeks an embedding of objects in a dimensional space such that inter-vector distances approximate pairwise object dissimilarities. Despite their popularity, MDS algorithms are sensitive to outliers, yielding grossly erroneous embeddings even if few outliers contaminate the available dissimilarities. This work introduces robust MDS ap- proaches exploiting the degree of sparsity in the outliers present. Links with compressive sampling lead to robust MDS solvers capable of coping with unstructured and structured outliers. The novel algorithms rely on a majorization-minimization approach to minimize a regularized stress function, whereby iterative MDS solvers involving Lasso and sparse group-Lasso operators are obtained. The resulting schemes identify outliers and obtain the desired embedding at computational cost comparable to that of their nonrobust MDS alternatives. The robust structured MDS al- gorithm considers outliers introduced by a sparse set of objects. In this case, two types of sparsity are exploited: i) sparsity of outliers in the dissimilarities; and ii) sparsity of the objects introducing outliers. Numerical tests on synthetic and real datasets illustrate the merits of the proposed algorithms. Index Terms—(Block) coordinate descent, (group) Lasso, multi- dimensional scaling, robustness, sparsity. I. INTRODUCTION M ULTIDIMENSIONAL scaling (MDS) broadly refers to exploratory data tools that find an embedding (a.k.a. configuration) of objects in a -dimensional vector space. The embedding is chosen such that inter-vector distances ap- proximate the given pairwise dissimilarities among the ob- jects, see e.g., [2], [9]. Originally, MDS was developed in psy- chology to visualize via two-dimensional maps perceptual rela- tionships among objects [21], [32]. Early applications of MDS in marketing aimed to position products in a perceptual map, and infer dimensions that explain, e.g., the features making a product more appealing [5], [26]. Recently, MDS has been suc- cessfully applied to areas ranging from high-dimensional data visualization to sensor network localization [3], [8]. Classical MDS uses the principal components of the double-centered Euclidean distance matrix to obtain the em- bedding when dissimilarities correspond to Euclidean distances [32]. Although able to perform well with exact distances, even a single “inconsistent” distance, hereafter termed outlier, can Manuscript received September 22, 2011; revised February 15, 2012 and April 21, 2012; accepted April 22, 2012. Date of publication May 03, 2012; date of current version July 10, 2012. The associate editor coordinating the re- view of this manuscript and approving it for publication was Dr. Konstantinos Slavakis. Work in this paper was in part supported by the AFOSR MURI Grant FA9550-10-1-0567. The authors are with the Electrical and Computer Engineering Depart- ment, University of Minnesota, Minneapolis, MN 55455 USA (e-mail: forer002@umn.edu; georgios@umn.edu). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TSP.2012.2197617 render the classical MDS solution of limited use. In particular, double centering spreads the effect of an outlier to the entire double-centered distance matrix, see e.g., [6]. An alternative to MDS, which can accommodate transformations on the dissimi- larities as well as missing data, relies on stress functions [2, ch. 11]. A popular algorithm to minimize the so-called raw stress is “scaling by majorizing a complicated function,” which is abbreviated as SMACOF [10]. The stress function is a weighted sum of squared-errors between dissimilarities and embedding inter-vector distances. Unfortunately, the dependency of stress functions on the least-squares (LS) criterion renders SMACOF and related stress-based MDS schemes sensitive to outliers [17]. Despite the popularity of MDS in various applications, dealing with outliers has been relegated to pre- and post-pro- cessing tasks, which remove “suspicious” dissimilarities based on “large” residual errors [9]. These two-step approaches to find the embedding and remove outliers have to decide when it is justified to discard dissimilarities. One of the first systematic approaches to tackle the outlier sensitivity of MDS resorted to tools from robust statistics for dissimilarities expressed as Euclidean distances [28]. The embedding was obtained by heuristically modified Newton–Raphson iterations solving a nonlinear system of equations. A criterion which exploited the known resilience to outliers of the -error norm was proposed to obtain a robust Euclidean embedding (REE) [6]. The resulting solvers were based on semidefinite program- ming and sub-gradient descent methods. Although rooted on a robust criterion, it was empirically observed that REE yields embeddings that underestimate the given dissimilarities, thus it further requires a robust scale estimate to properly adjust the embedding. In the context of sensor networks, the Huber function was employed to find an embedding corresponding to sensor positions [20]. The resulting algorithm used a two-level majorization-minimization (MM) strategy, where each min- imization step of the first level iteration was solved through an MM algorithm. As acknowledged in [20], this nested MM structure slows down convergence. The motif behind the robust MDS algorithms developed in this work is the degree of sparsity present in the outliers contaminating the dissimilarity data. Leveraging this knowl- edge, robust MDS based on the least-trimmed squares (LTS) criterion is proposed in Section II. It is shown that robust MDS via LTS is equivalent to a regularized LS approach stemming from a dissimilarity model that explicitly accounts for outliers. The regularization term corresponds to the -norm and establishes a link between robust MDS and the area of compressive sampling [4], [33]. Capitalizing on this link, the -norm is relaxed to its closest convex approximation, namely the -norm. An MM-based iterative algorithm is de- veloped in Section III, which involves closed-form solutions of 1053-587X/$31.00 © 2012 IEEE