IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 63, NO. 11, JUNE 1, 2015 2915 On the Characterization of -Compressible Ergodic Sequences Jorge F. Silva, Member, IEEE, and Milan S. Derpich, Member, IEEE Abstract—This work offers a necessary and sufficient condition for a stationary and ergodic process to be -compressible in the sense proposed by Amini, Unser and Marvasti [“Compressibility of deterministic and random infinity sequences,” IEEE Trans. Signal Process., vol. 59, no. 11, pp. 5193–5201, 2011, Def. 6]. The condition reduces to check that the -moment of the invariant distribution of the process is well defined, which contextualizes and extends the result presented by Gribonval, Cevher and Davies in [“Compress- ible distributions for high-dimensional statistics,” IEEE Trans. Inf. Theory, vol. 58, no. 8, pp. 5016–5034, 2012, Prop. 1]. Furthermore, for the scenario of non- -compressible ergodic sequences, we pro- vide a closed-form expression for the best -term relative approx- imation error (in the -norm sense) when only a fraction (rate) of the most significant sequence coefficients are kept as the sequence- length tends to infinity. We analyze basic properties of this rate-ap- proximation error curve, which is again a function of the invariant measure of the process. Revisiting the case of i.i.d. sequences, we completely identify the family of -compressible processes, which reduces to look at a polynomial order decay (heavy-tail) property of the distribution. Index Terms—Asymptotic analysis, best -term approximation error analysis, compressed sensing, compressibility of infinite se- quences, compressible priors, ergodic processes, heavy-tail distri- butions. I. INTRODUCTION D EFINING notions of compressibility for a stochastic process, meaning that with high probability realizations of the process can be well-approximated in some sense by its best -term sparse version [3], has been a recent topic of active research [1], [2], [4]–[6]. Quantifying compressibility for random sequences and the identification of compressible and sparse distributions (priors) are relevant problems consid- ering the recent development of the compressed sensing theory [7]–[9] and its applications. These results can play an important role in regression [10], signal reconstruction (for instance in Manuscript received July 28, 2014; revised January 11, 2015; accepted March 13, 2015. Date of publication April 02, 2015; date of current version May 05, 2015. The associate editor coordinating the review of this manuscript and ap- proving it for publication was Prof. Fancesco Verde. This material is based on work supported by grants of CONICYT-Chile, Fondecyt Grant 1140840 and the Advanced Center for Electrical and Electronic Engineering (AC3E), Basal Project FB0008. In addition, the work of M. S. Derpich was partially sup- ported by CONICYT Fondecyt Grant 1140384. (Corresponding Author: Jorge F. Silva.) J. F. Silva is with the Department of Electrical Engineering, Information and Decision Systems Group, University of Chile, 412-3 Santiago, Chile (e-mail: josilva@ing.uchile.cl). M. S. Derpich is with the Department of Electronic Engineering, Universidad Técnica Federico Santa María, Valparaíso 2390123, Chile (e-mail: milan.der- pich@usm.cl). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TSP.2015.2419183 the classical compressed sensing setting [2, Th. 2]), inference, and decision-making problems [11], [12]. One important case is defining such a compressibility notion for i.i.d. processes where the probability measure is equipped with a density function 1 [1], [2]. In this context, realizations of the process are non-sparse (almost surely), and conventional ways of defining compressibility for finite dimensional signals, based on the power-law decay of the best -term approximation error (or sequences that belong to the weak- ball), are not applicable either, as shown in [1], [2]. Motivated by this problem, Amini et al. [1] and Gribonval et al. [2] have introduced new definitions for compressible random sequences. These notions are not based on the typical absolute approximation error decay pattern of the signals, but on a rela- tive -best -term approximation error behavior. In particular, Amini et al. [1] formally define the concept of -compressible process (details in Section II below). This new definition pro- vides a meaningful way of categorizing i.i.d. random sequences (and their distributions), in terms of the probability that almost all the -relative energy of the process is concentrated in an arbitrarily small sub-dimension of the coordinate domain, as the block-length tends to infinity. Under this context, they pro- vide two important results using the theory of order statistics [1]. First of all, [1, Theorem 3] shows that a concrete family of i.i.d. heavy-tail distributions is -compressible (including the generalized Pareto, Students’s and log-logistic), while on the other side, [1, Theorem 1] demonstrates that families with expo- nentially decaying tails (such as Gaussian, Laplace, generalized Gaussian) are not -compressible. Therefore, it is interesting to ask about the compressibility of i.i.d processes not considered in that analysis. In this direction, we highlight the work of Gri- bonval et al. [2], which under an alternative notion of relative -compressibility (involving almost sure convergences instead of convergence in measure, which was the criterion adopted in [1]) and a different analysis setting (fixed-rate instead of the variable rate used in [1]), elaborates an exact dichotomy be- tween compressible and non-compressible i.i.d. sequences. This raises the question of whether it is possible to connect Amini et al. [1] -compressibility with the more refined almost sure (a.s.) convergence analysis of the best -term relative approx- imation error in [2, Prop. 1], with the idea of completing the analysis of [1, Ths. 1 and 3]. To address this question, we extend the analysis from i.i.d. sequences to stationary and ergodic processes. In this broader setting, the main result (Theorem 1) provides a necessary and sufficient condition for a stationary and ergodic process to be -compressible (in the sense of Amini et al. [1, Def. 6]), for 1 The probability is absolutely continuous with respect to the Lebesgue mea- sure [13]. 1053-587X © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.