3D segmentation of cell boundaries from whole cell cryogenic electron tomography volumes Farshid Moussavi a , Geremy Heitz a , Fernando Amat a , Luis R. Comolli c , Daphne Koller b, * , Mark Horowitz a,b, * a Department of Electrical Engineering, Stanford University, Stanford, CA, USA b Department of Computer Science, Stanford University, Stanford, CA, USA c Life Sciences Division, Lawrence Berkeley National Labs, Berkeley, CA, USA article info Article history: Received 21 October 2009 Received in revised form 14 December 2009 Accepted 16 December 2009 Available online 24 December 2009 Keywords: Electron tomography Biomedical image processing Segmentation Object detection Computer vision Probabilistic inference abstract Cryogenic electron tomography (cryo-ET) has gained increasing interest in recent years due to its ability to image whole cells and subcellular structures in 3D at nanometer resolution in their native environ- ment. However, due to dose restrictions and the inability to acquire high tilt angle images, the recon- structed volumes are noisy and have missing information. Thus, features are unreliable, and precision extraction of the cell boundary is difficult, manual and time intensive. This paper presents an efficient recursive algorithm called BLASTED (Boundary Localization using Adaptive Shape and Texture Discovery) to automatically extract the cell boundary using a conditional random field (CRF) framework in which boundary points and shape are jointly inferred. The algorithm learns the texture of the boundary region progressively, and uses a global shape model and shape-dependent features to propose candidate bound- ary points on a slice of the membrane. It then updates the shape of that slice by accepting the appropriate candidate points using local spatial clustering, the global shape model, and trained boosted texture clas- sifiers. The BLASTED algorithm segmented the cell membrane over an average of 93% of the length of the cell in 19 difficult cryo-ET datasets. Ó 2009 Elsevier Inc. All rights reserved. 1. Introduction Electron tomography is the process of taking multiple electron micrographs of a biological sample (usually a whole cell or subcel- lular structure) from different angles, and reconstructing a 3D im- age of the sample (Koster and Klumperman, 2003; Baumeister, 2002). In cryogenic electron tomography (cryo-ET), the sample is first flash frozen to retain the biological and molecular structure as much as possible, enabling the study of macromolecular cell fea- tures such as cell membrane, surface layer (S-layer) components, ribosomes, filaments, and cytoskeletal structures (Gan et al., 2008; Ortiz et al., 2006) in their native natural environment. One slice of a typical electron tomogram taken from a dividing Caulob- acter crescentus bacterium is shown along with its segmented boundary surface in Fig. 1. Unfortunately, often after a reconstruction is obtained, a phe- nomenon of interest cannot be seen. For many recent results it is common to take dozens of tomograms before finding one that is useful. Therefore high throughput tomography is becoming essen- tial for high research productivity. Recent advances (Zheng et al., 2007; Mastronarde, 2005; Heymann and Belnap, 2007; Amat et al., 2007; Lawrence et al., 2006; Suloway et al., 2005) have auto- mated the acquisition and reconstruction of the tomograms, en- abling the generation of hundreds of tomograms in a few weeks instead of years. The bottleneck is now in the postprocessing. A common first postprocessing step is the segmentation of the cell boundary, which reduces the size of the volume in which any search for cytoplasmic features (such as ribosomes, filaments, cytoskeletal features) needs to operate. In the absence of reliable automatic methods, an expert user today can spend significant amounts of time (ranging from hours to days) manually extracting this boundary by clicking on thousands of points. The goal of this work is to automatically segment the cell boundary in electron tomograms, removing this bottleneck. An adequate solution to this task must address several signifi- cant challenges arising from the nature of the cryo-ET technology. Cryogenically prepared samples can tolerate little total electron dose before being damaged, which translates to a low signal to noise ratio throughout the micrographs and the 3D reconstruction. Furthermore, as the sample is rotated around a tilt axis, electron beams must traverse a thickness of ice which increases with 1= cosðcÞ, where c is the rotation angle, making it difficult to obtain micrographs at high angles. The lack of high angle projections acts as an orientation filter, referred to as the ‘missing wedge’; it implies that the 3D reconstruction will be missing information. 1047-8477/$ - see front matter Ó 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.jsb.2009.12.015 * Corresponding authors. Address: Department of Computer Science, Stanford University, Stanford, CA, USA. E-mail addresses: koller@cs.stanford.edu (D. Koller), horowitz@stanford.edu (M. Horowitz). Journal of Structural Biology 170 (2010) 134–145 Contents lists available at ScienceDirect Journal of Structural Biology journal homepage: www.elsevier.com/locate/yjsbi