Source Description Cost Hossein Kakavand and Abbas El Gamal Department of Electrical Engineering Stanford University Stanford, CA, 94305, USA E-mail: hossein@stanford.edu; abbas@stanford.edu Abstract— The paper considers the source description problem with average distortion and per-symbol reproduction cost con- straints. The source description cost-distortion function is then defined as the minimum of a weighted sum of the rate and the expected per-symbol reproduction cost subject to average distortion constraint. This function is evaluated for (i) binary source, Hamming loss and reproduction cost of 0 for a 0 and 1 for 1, (ii) Gaussian source, squared error distortion, and average power constraint on reproduction, and (iii) Gaussian Wyner-Ziv source coding with side information setting and average power constraint on reproduction. The results are compared to the description cost in the classical case. I. I NTRODUCTION Consider Shannon’s classical source description, or rate distortion, problem for an i.i.d. source X ∼ p(x) with alphabet X as depicted in Figure 1. The encoder assigns an index i(x n ) ∈ [1, 2 nR ] to each outcome and the decoder generates a description ˆ x n (i) ∈ ˆ X n for each index i ∈ [1, 2 nR ], where ˆ X is the reproduction alphabet. The problem is to find the minimum rate R(D) needed to achieve a desired average distortion D with respect to a per-symbol distortion measure d : X× ˆ X→ [0, ∞). In [2] Shannon showed that R(D)= inf p(ˆ x|x):E[d(X, ˆ X)] ≤ D I (X; ˆ X). (1) This result was later refined and extended by many other researchers (e.g., see [1]). X n ˆ X n I ∈{1,..., 2 nR } Decoder Encoder Fig. 1. Source description setting. In Shannon’s setting, the only cost associated with generat- ing the source description is the number of bits required per source symbol. In other words, only the cost of generating, transmitting, or storing the index is considered. In practical lossy compression scenarios such as analog to digital conver- sion, image, video, and speech compression, and collaborative signal processing over a sensor network, there is also a cost as- sociated with the generation of the description from the index. In most analog to digital converter architectures, a sequence of bits, which can be viewed as the index in the rate distortion setting, is first generated and then processed to obtain the description. In image, speech, and video compression systems, the encoder produces a compressed version of the raw data (the index) to be transmitted or stored. The compressed data is then processed by the decoder to produce a description suitable for human or machine use. In a sensor network, the nodes typically have very limited energy and throughput budgets, and thus data from each sensor must be compressed locally and then decoded by one or several other nodes for performing collaborative processing. In all of these scenarios, there is a cost associated with the generation of the description from the index. In some scenarios, such as analog to digital conversion, the encoder and decoder reside within the same system and the total system cost must then include both the cost of generating the index as well as the cost of generating the description. In other scenarios, the encoder and decoder reside in different systems, but the design of the compression algorithm must consider the tradeoff between the cost of encoding and that of decoding. In this paper we propose to incorporate the cost of generat- ing the description from the index into the source description problem via a per-symbol reproduction cost. Since any decoder must ultimately “output” the description, it must at a minimum incur such a cost. This formulation provides a mechanism for including the cost of decoding while avoiding the difficulty of modelling its actual cost, which would require finding an implementation independent cost of computing. In the next section we introduce the problem of source description with per-symbol reproduction cost and define the description cost distortion function. In Section III, we evaluate this function for a binary source, Hamming distortion measure and a reproduction function that assigns an unequal weight to 1 than 0. In Section IV, we evaluate this function for a Gaussian source, squared error distortion, and average power constraint on reproduction. In Section V, we evaluate it for the Gaussian Wyner-Ziv setting. II. DESCRIPTION COST DISTORTION FUNCTION Consider the source description problem with i.i.d. source p(x), x ∈X , reproduction alphabet ˆ X and distortion mea- sure d. Additionally assume a per-symbol reproduction cost function b : ˆ X→ [0, ∞). Define a (2 nR ,n) rate distortion- cost-code to consist of an encoding function that maps each sequence x n ∈X n into an index i(x n ) ∈ [1, 2 nR ] and a decoding function that maps each index i ∈ [1, 2 nR ] into a description ˆ x n ∈ ˆ X n . The distortion associated with the