IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 4, APRIL 2012 1863 Multiview Deblurring for 3-D Images from Light-Sheet-Based Fluorescence Microscopy Maja Temerinac-Ott, Olaf Ronneberger, Peter Ochs, Wolfgang Driever, Thomas Brox, and Hans Burkhardt Abstract—We propose an algorithm for 3-D multiview deblur- ring using spatially variant point spread functions (PSFs). The algorithm is applied to multiview reconstruction of volumetric microscopy images. It includes registration and estimation of the PSFs using irregularly placed point markers (beads). We formu- late multiview deblurring as an energy minimization problem subject to L1-regularization. Optimization is based on the regu- larized Lucy–Richardson algorithm, which we extend to deal with our more general model. The model parameters are chosen in a profound way by optimizing them on a realistic training set. We quantitatively and qualitatively compare with existing methods and show that our method provides better signal-to-noise ratio and increases the resolution of the reconstructed images. Index Terms—Deblurring with spatially variant point spread function (PSF), deconvolution, fluorescence microscopy, L1-reg- ularization, PSF estimation, registration with irregularly placed point markers, tomography, 3-D images. I. INTRODUCTION B IOMEDICAL research and developmental biology de- mand imaging techniques that allow large specimens to be investigated. This is achieved by modern fluorescence mi- croscopes, which can record volumetric images of small model organisms at a high resolution. Because of the decrease in the signal in -direction due to absorption, larger specimens are recorded from different views in order to obtain high-quality images of each part. These views need to be fused in order to produce one high-resolution image of the whole sample. We focus here on images recorded with single-plane illumination microscopes (SPIMs) [1]. Compared with confocal micro- scopes, which scan over single 3-D points of the probe, this technique can record images quite fast as a whole plane is recorded in a single step. The recording speedup is achieved by using a light sheet for illumination and optical sectioning. Manuscript received June 08, 2011; revised October 21, 2011 and November 29, 2011; accepted December 16, 2011. Date of publication December 23, 2011; date of current version March 21, 2012. This work was supported in part by the Excellence Initiative of the German Federal Governments (EXC 294) and in part by SFB 592. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Sina Farsiu. M. Temerinac-Ott, O. Ronneberger, P. Ochs, T. Brox, and H. Burkhardt are with the Chair of Pattern Recognition and Image Processing, Department of Computer Science, University of Freiburg, 79104 Freiburg, Germany, and also with the BIOSS Centre for Biological Signalling Studies, Albert-Ludwigs-Uni- versity of Freiburg, 79108 Freiburg, Germany (e-mail: temerina@informatik. uni-freiburg.de). W. Driever is with the Chair of Developmental Biology, Institute of Bi- ology, University of Freiburg, 79104 Freiburg, Germany, and also with the BIOSS Centre for Biological Signalling Studies, Albert-Ludwigs-University of Freiburg, 79108 Freiburg, Germany. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIP.2011.2181528 However, the SPIM setting leads to blurring with a point spread function (PSF) that varies within the light sheet. This effect must be corrected to achieve optimal high-resolution results. Consequently, a powerful system for 3-D multiview reconstruc- tion of SPIM recordings requires registration of multiple 3-D images from different views, estimation of the unknown and varying PSF, and multiview deblurring based on the registered images and estimated PSF (see Fig. 1). State-of-the-art methods for the deblurring of microscopical data only consider single views. The PSF is either assumed to be invariant and known [2], or it is estimated during image restora- tion [3]–[5]. The problem of joint deblurring of multiple record- ings with different PSFs has been so far only tackled for 2-D images [6], [7]. To our knowledge, the only extension of mul- tiview deblurring to 3-D has been performed in [8] on SPIM images. There, the PSF is assumed to be invariant and known beforehand. For the multiview reconstruction of SPIM images, the standard method [9] does not involve deblurring by a PSF and is solely based on a pixel-based weighting of the single im- ages. The existing methods do not present a satisfactory solution to the multiview deblurring problem, since they either do not model multiple views or do not assume a varying PSF model. We formulate a multiview reconstruction model with a varying PSF. To ensure that the algorithm can be applied to any biological object, irregularly and sparsely distributed fluo- rescent small spherical objects, which are the so-called beads, are added to the recording volume as point markers. This puts registration of multiple views as a point cloud registration problem. We have presented a solution to this problem based on group integration (GI) in [10] and employ it in the current system. Moreover, the precise estimation of the spatially variant PSF is facilitated by the beads since their size and shape is known. The beads are only available outside the probe, i.e., the PSF must be interpolated in the probe volume, which we achieve by straightforward kernel regression. The main contribution of this paper is a whole reconstruc- tion framework for 3-D multiview microscopic images with the focus on the multiview 3-D deblurring with varying PSFs using a modified Lucy–Richardson (LR) algorithm. An earlier version of the deblurring algorithm without an underlying variational model has been presented at a conference [11]. In this paper, apart from the variational model, we also present a way to opti- mize the model parameters on a realistic synthetic training data set and perform an extended quantitative comparison with ex- isting methods on synthetic and real SPIM data sets. The variational model builds on a maximum likelihood for- mulation of the problem, which is optimized using the expec- tation-maximization (EM) algorithm [12]–[14]. For the single- 1057-7149/$26.00 © 2011 IEEE