Perturbation Stability of Various Coherent Riesz Families Werner Kozek a , G¨otz Pfander b , and Georg Zimmermann *b a Siemens AG, Information and Communication Networks Hofmannstr. 51, D–81359 Munich, Germany b Institut f¨ ur Angewandte Mathematik und Statistik Universit¨at Hohenheim, D–70593 Stuttgart, Germany ABSTRACT We compare three types of coherent Riesz families (Gabor systems, Wilson bases, and wavelets) with respect to their perturbation stability under convolution with elements of a family of typical channel functions. This problem is of key relevance in the design of modulation signal sets for digital communication over time–invariant channels. Upper and lower bounds on the orthogonal perturbation are formulated in terms of spectral spread and temporal support of the prototype, and by the approximate design of worst case convolution kernels. Among the considered bases, the Weyl–Heisenberg structure which generates Gabor systems turns out to be optimal. Keywords: Perturbation stability, coherent families, Gabor systems, Wilson bases, wavelets, channel function 1. INTRODUCTION A coherent function system is built from a finite number of prototype functions by the group action of unitary opera- tors such as translation, modulation and/or scaling. The inherent structure of such systems leads to computationally efficient design and implementation of frames or Riesz bases. The most prominent coherent function systems are wavelet and Gabor systems. Both structures are potential candidates in the two fundamental applications of modern digital communication: • Source coding (signal compression): The coherent function system conveyes the transform step which aims at decorrelating the data prior to quantization. In near-to-lossless compression completeness is a must, hence the function system is required to be a frame. • Channel coding (signal transmission): The channel input signal is synthesized as a linear combination of certain basis functions whose coefficients are bearing the information. Here, injectivity of this synthesis mapping is crucial, therefore one actually wants to use a Riesz basis for some closed subspace of the underlying Hilbert space (on which the channel acts as a linear operator). In both applications, the performance is reflected by an operator diagonalization problem; the operator corresponds either to the correlation of the source or to the action of the channel, respectively. Exact diagonalization is unrealistic because the a priori knowledge of the underlying operator is incomplete, and even if we had this prior knowledge, the resulting eigenbases are unstructured and do not satisfy practical side constraints (such as finite support). We shall concentrate on channel coding. As bases, we consider shift-invariant Riesz systems g k,l defined by g k,l (x)= g l (x−ak), k ∈ Z , l =0, 1,...,N −1 , (1) where a> 0 is the time shift, each g l has support of length at most a (because of the latency constraints), and the family has one of the following specific structures: • Gabor or Weyl–Heisenberg systems 1 correspond to a rectangular tiling of the time–frequency plane, the g l are modulated versions of a prototype function g 0 : g l (x)= g 0 (x)e 2πiblx . Note that in order to have existence of Riesz families, one necessarily has b ≥ 1/a. * Correspondence: Email: gzim@uni-hohenheim.de Tel.: +49 / 711 / 459–2449 Fax: +49 / 711 / 459–3030