Optimal Data Compression and Filtering: the Case of Infinite Signal Sets Anatoli Torokhti and Phil Howlett Abstract—We present a theory for optimal filtering of infinite sets of random signals. There are several new distinctive features of the proposed approach. First, we provide a single optimal filter for processing any signal from a given infinite signal set. Second, the filter is presented in the special form of a sum with p terms where each term is represented as a combination of three operations. Each operation is a special stage of the filtering aimed at facilitating the associated numerical work. Third, an iterative scheme is implemented into the filter structure to provide an improvement in the filter performance at each step of the scheme. The final step of the concerns signal compression and decompression. This step is based on the solution of a new rank-constrained matrix approximation problem. The solution to the matrix problem is described in this paper. A rigorous error analysis is given for the new filter. Keywords—stochastic signals, optimization problems in signal processing. I. I NTRODUCTION A. Motivation I N this paper, we consider extensions of known approaches to optimal filtering based on the Wiener idea 1 . We present a theory for a new nonlinear filter which processes infinite sets of random signals. The filter is constructed via an iterative scheme that provides a signal processing improvement with each step. The filter provides simultaneous signal filtering and compression and the subsequent decompression (reconstruc- tion). There has been significant attention in the literature to filters that process finite sets of random signals but it seems that a filter which is able to process infinite sets of random signals has not been developed. The filter presented in this paper is designed specifically to process infinite sets of random signals. For the case of finite sets of random signals, we show that our filter leads to a lower computational load and better accuracy than the known filters; the improved accuracy is due to the special iteration procedure incorporated into the filter structure (see Section III-E2). There are three motivations for the proposed method which we now describe. 1) First motivation: infinite sets of signals: Most of the literature on Wiener-like filtering provides an optimal filter for an individual input signal given by a finite random vector 2 . This means that if we wish to transform an infinite set Y = {y (1) , y (2) ,..., y (N) ,...} of input vector signal into an Anatoli Torokhti and Phil Howlett are with the School of Mathematics and Statistics, University of South Australia, Mawson Lakes, SA 5095, Australia, email: anatoli.torokhti@unisa.edu.au and phil.howlett@unisa.edu.au, 1 Some references on Wiener-like filtering can be found in [8], [12], [16], [18], [19]. 2 We say a random vector x is finite if each realization x = x(ω) has a finite number of scalar components. infinite set X = {x (1) , x (2) ,..., x (N) ,...} of output vector signals using a Wiener-like approach then we have to find a set of corresponding Wiener filters {F (1) , F (2) ,..., F (N) ,...} so that each representative F i of the filter set relates to a representative y (i) of the signal-vector set Y . Therefore such a filter cannot be applied if X and Y are infinite sets of signals. Moreover, in some situations, a recognizer must be used that will determine to which of the filters {F (1) , F (2) ,..., F (N) ,...} each component from Y should be directed. Note that even in the case when Y and X are finite sets, Y = {y (1) , y (2) ,..., y (N) } and X = {x (1) , x (2) ,..., x (N) } where Y and X can be represented as finite vectors, the Wiener approach leads to computation of large covariance matrices. Indeed, if each y i has n components and each x i has m components then the Wiener approach leads to computation of a product of an mN × nN matrix and an nN × nN matrix and computation of an nN × nN pseudo-inverse matrix [18]. This requires O(2mn 2 N 3 ) and O(22n 3 N 3 ) flops, respectively [7]. If m, n and N are sufficiently large then the computational work associated with this approach becomes unreasonably hard. To avoid such drawbacks here, we here study an approach that allows us to use only one filter to process any signal from the infinite set Y . The first question we address in the paper is as follows. Let X and Y be infinite sets of signals. How should we construct a single optimal filter F : Y → X which can be applied to each pair of signals (x, y) ∈ X × Y and which, moreover, transforms each y to a corresponding x with associated minimal error? Surprisingly, perhaps, the answer is based firstly, on an equivalent alternative signal representation in a different space and secondly, on the use of a special norm (3) in the statement of the problem. The dual representation means that x is considered as a single signal in one representation, and on the other hand, as an infinite set of signals in the other, original, representation. A detailed explanation is given in Section VI. Examples of different special cases of the norm (3) used in our statement of the problem are presented in Section VI. The answer for the first question is provided in Sections II-A, II-C, in Theorems 3 and 4, and in Section III-E. The special norm is given by (3) below. 2) Second motivation: improvement in the filter perfor- mance: The performance of filters used for data filtering, compression and subsequent reconstruction, is characterized by the accuracy, the compression ratio and the related com- putational load. The Karhunen-Lo` eve filter (KLF) [13], [14], World Academy of Science, Engineering and Technology International Journal of Electronics and Communication Engineering Vol:2, No:11, 2008 2604 International Scholarly and Scientific Research & Innovation 2(11) 2008 scholar.waset.org/1307-6892/11167 International Science Index, Electronics and Communication Engineering Vol:2, No:11, 2008 waset.org/Publication/11167