Sparse Signal Recovery in the Presence of Intra-Vector and Inter-Vector Correlation Bhaskar D. Rao Dept. of Electrical and Computer Eng. University of California, San Diego La Jolla, 92093-0407, USA Email: brao@ucsd.edu Zhilin Zhang Dept. of Electrical and Computer Eng. University of California, San Diego La Jolla, 92093-0407, USA Email: z4zhang@ucsd.edu Yuzhe Jin Microsoft Research One Microsoft Way Redmond, WA 98052, USA Email: yuzjin@microsoft.com Abstract— This work discusses the problem of sparse signal recovery when there is correlation among the values of non- zero entries. We examine intra-vector correlation in the context of the block sparse model and inter-vector correlation in the context of the multiple measurement vector model, as well as their combination. Algorithms based on the sparse Bayesian learning are presented and the benefits of incorporating correlation at the algorithm level are discussed. The impact of correlation on the limits of support recovery is also discussed highlighting the different impact intra-vector and inter-vector correlations have on such limits. I. I NTRODUCTION The problem of sparse signal recovery has many potential applications [1], [2] and has received much attention in recent years with the development of compressed sensing (CS) [3], [4]. The general Multiple Measurement Vector (MMV) model is given by [5] Y = ΦX + V. (1) Here Y  [Y ·1 , ··· , Y ·L ] ∈ R N×L is an available mea- surement matrix consisting of L measurement vectors. Φ ∈ R N×M (N ≪ M ) is a known matrix, and any N columns of Φ are linearly independent. X  [X ·1 , ··· , X ·L ] ∈ R M×L is an unknown and full column-rank matrix of interest. A key assumption here is that X has only a few non-zero rows. V is a noise matrix. The special case of L =1 is the widely studied Single Measurement Vector (SMV) problem in CS and in this context we use x to denote the vector of interest. II. STRUCTURE IN X In the basic SMV and MMV models no additional as- sumptions are usually made. However, in many applications additional structure on X is available and we now discuss a few of them. (1) For the SMV problem, in contrast to the usual assump- tions that the locations of non-zero entries are independently and uniformly distributed, some dependency in the locations is assumed [6]–[8]. Incorporating this structure is important from an application point of view and this structure can be exploited to improve the performance of algorithms. (2) In the SMV problem a widely studied structure is block/group structure [9], [10]. With this structure, x can be viewed as a concatenation of g blocks, i.e. x =[x 1 , ··· ,x d 1    x T 1 , ··· ,x d g-1 +1 , ··· ,x d g    x T g ] T (2) where d i (∀i) are not necessarily the same. Among the g blocks, only k blocks are nonzero, where k ≪ g. This can be viewed as a special case of modeling the distribution of the locations of the non-zero entries, but is worthy of special attention because of its application potential. In general, no ad- ditional assumption is made about the entries in each nonzero block. Motivated by applications, it appears reasonable to assume that the entries in each non-zero block are correlated [11], [12]. We refer to this as intra-block correlation and will discuss it in detail in Section III-A. (3) In the basic MMV problem, the typical assumption made is that the vectors in X share a common sparsity profile. This leads to non-zero rows in X. One can impose additional structure. One possibility could be dependency in the locations of the non-zero rows. And the other is correlation between the entries in each of the non-zero rows [13], [14]. We refer to the correlation as inter-vector correlation and will discuss it in Section III-B. (4) One can combine the above-mentioned two types of structure and consider the problem of block sparsity in the MMV problem. This leads to the consideration of correlated non-zero blocks of rows in X. The challenge in this context is efficiently modeling and estimating the correlation structure. (5) The time-varying sparsity model is a natural extension of the MMV model [15]–[17]. It considers the case when the support of each column of X is time-varying. The time-varying structure calls for modeling both the variation in the number and locations of the non-zero entries as well as the correlation of the non-zero entries. III. I NTRA-VECTOR AND I NTER-VECTOR CORRELATION A. Intra-Vector Correlation For the SMV problem with the block structure (2), a number of algorithms have been proposed, such as the Group Lasso [9]. But few consider correlation within each block x i (∀i), namely the intra-block correlation.