Improved POCS-based image reconstruction from irregularly-spaced samples Ryszard Stasi´ nski Institute of Electronics and Telecommunications, Technical University of Pozna´ n Piotrowo 3A, PL-60-965 Pozna´ n, Poland, rstasins@et.put.poznan.pl Janusz Konrad Department of Electrical and Computer Engineering, Boston University 8 Saint Mary’s Street, Boston, MA 02215, USA, jkonrad@bu.edu In Proc. XI European Signal Proc. Conf., EUSIPCO-2002, Sep. 3–6, 2002, Toulouse, France c  2002 EURASIP. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the EURASIP. ABSTRACT This paper presents an enhanced POCS-based (projec- tion onto convex sets) method for the reconstruction of a regularly-sampled image from its irregularly-spaced samples. Such a reconstruction is often needed in image processing and coding, for example when using motion compensation. The proposed approach applies two op- erators sequentially: bandwidth limitation and sample substitution, and is based on our earlier work. The con- tribution of this paper is a new, simpler implementation of the algorithm that allows for faster convergence, and provides better performance, although at the cost of in- creased memory requirements. 1 Introduction Interpolation of intensity from a set of known samples is a common task in image processing and coding. The grids of unknown and known samples can each be de- fined as a regular (periodic) or irregular sampling struc- ture (grid). There are 4 scenarios for sampling grid com- binations of unknown/known samples: 1. regular→regular - simplest case where interpolation filters are space-invariant; for example, interpolat- ing filters in typical image up-conversion, 2. regular→irregular - more difficult case where inter- polation filters are space-variant since samples to be recovered have varying positions with respect to the regularly-spaced known samples; for exam- ple, bilinear or bicubic interpolation [1] in backward motion compensation in video coding, 3. irregular→regular - still more difficult case where interpolation filters are space-variant but may not be easily specified in general case; for example, for- ward motion-compensation in advanced video cod- ing/interpolation, 4. irregular→irregular - the most general case for which applications have not clearly emerged yet. The first two cases have been extensively treated in the literature and have found numerous practical applications in image processing and coding. The irregular→regular interpolation has been explored to a lesser degree. The primary reason for this are difficul- ties associated with the extension of Shannon’s sampling theory to signals defined over irregular sampling grids; alternative methods must be found to reconstruct or ap- proximate the original continuous signal. Although some results on the reconstruction of band- limited functions from their irregularly-spaced samples are available (e.g., [2]), their usefulness in the case of motion-compensated video coding or interpolation is quite limited; theoretically derived constraints on the maximum spacing of irregular samples under the per- fect reconstruction condition cannot be satisfied in prac- tice by arbitrarily-distributed image samples after mo- tion compensation. By relaxing the perfect reconstruc- tion condition, other methods were proposed such as the polynomial interpolation or iterative reconstruction [3]. In this paper, we extend our earlier approach to irregular→regular interpolation [5, 6] that is based on projections onto convex sets (POCS) [4]. Our new method differs in implementation that is simpler, con- verges faster and provides better PSNR performance, although at the cost of increased memory requirements. 2 Proposed approach Let g = {g(x), x =(x, y) T ∈ R 2 } be a continuous 2-D projection of the 3-D world onto an image plane and let g Λ = {g(x), x ∈ Λ} be a discrete image obtained from g by sampling over a lattice Λ [7]. Let’s assume that g is band-limited, i.e., G(f )= F{g}=0 for f ∈ Ω where F is the Fourier transform, f =(f 1 ,f 2 ) T ∈ R 2 is a frequency vector and Ω ⊂ R 2 is the spectral sup- port of g. If the lattice Λ satisfies the multi-dimensional Nyquist criterion [7], the Shannon sampling theory al- lows to perfectly reconstruct g from g Λ . However, in the case of irregular sampling the theory is not applicable. Therefore, the general goal is to develop a method for the reconstruction of g from an irregular set of samples g Ψ = {g(x i ), x i ∈ Ψ ⊂ R 2 ,i =1, ..., K}, where Ψ is an irregular sampling grid.