Automatic particle selection: results of a comparative study Yuanxin Zhu, a Bridget Carragher, a Robert M. Glaeser, b Denis Fellmann, a Chandrajit Bajaj, c Marshall Bern, d Fabrice Mouche, a Felix de Haas, e Richard J. Hall, f David J. Kriegman, g Steven J. Ludtke, h Satya P. Mallick, g Pawel A. Penczek, i Alan M. Roseman, j Fred J. Sigworth, k Niels Volkmann, l and Clinton S. Potter a, * a Center for Integrative Molecular Biosciences and Department of Cell Biology, The Scripps Research Institute, La Jolla, CA 92037, USA b Department of Molecular and Cell Biology, University of California, Berkeley, CA 94720, USA c Department of Computer Sciences, University of Texas at Austin, Austin, TX 78712, USA d Computer Sciences Lab, Palo Alto Research Center, Palo Alto, CA 94304, USA e Division of Electron Optics, FEI Company, Eindhoven, The Netherlands f Department of Biological Sciences, Imperial College London, London SW7 2AY, UK g Department of Computer Science and Engineering, University of California, San Diego, CA 92093, USA h National Center for Macromolecular Imaging, Baylor College of Medicine, Houston, TX 77030, USA i Department of Biochemistry and Molecular Biology, University of Texas-Houston Medical School, Houston, TX 77225, USA j MRC Laboratory of Molecular Biology, Hills Road, Cambridge CB2 2QH, UK k Department of Cellular and Molecular Physiology, Yale University School of Medicine,New Haven, CT 06520, USA l The Burnham Institute, La Jolla, CA 92037, USA Received 15 August 2003 Abstract Manual selection of single particles in images acquired using cryo-electron microscopy (cryoEM) will become a significant bottleneck when datasets of a hundred thousand or even a million particles are required for structure determination at near atomic resolution. Algorithm development of fully automated particle selection is thus an important research objective in the cryoEM field. A number of research groups are making promising new advances in this area. Evaluation of algorithms using a standard set of cryoEM images is an essential aspect of this algorithm development. With this goal in mind, a particle selection ‘‘bakeoff’’ was included in the program of the Multidisciplinary Workshop on Automatic Particle Selection for cryoEM. Twelve groups participated by submitting the results of testing their own algorithms on a common dataset. The dataset consisted of 82 defocus pairs of high- magnification micrographs, containing keyhole limpet hemocyanin particles, acquired using cryoEM. The results of the bakeoff are presented in this paper along with a summary of the discussion from the workshop. It was agreed that establishing benchmark particles and using bakeoffs to evaluate algorithms are useful in promoting algorithm development for fully automated particle selection, and that the infrastructure set up to support the bakeoff should be maintained and extended to include larger and more varied datasets, and more criteria for future evaluations. Ó 2003 Elsevier Inc. All rights reserved. Keywords: Electron microscopy; Single-particle reconstruction; Automatic particle selection; Image processing; Pattern recognition 1. Introduction Selection of individual particles from digitized elec- tron micrographs begins to represent a labor-intensive bottleneck in single-particle cryo-electron microscopy (cryoEM) when the size of the dataset that is needed starts to exceed a few tens of thousand molecular im- ages. The automation of particle selection has been a topic of interest for many years (for a review, see, Nicholson and Glaeser, 2001). Apart from the task of selection of images of spherical virus particles at rela- tively high defocus, computer algorithms alone have not been as effective as most users wish them to be. As a * Corresponding author. Fax: 1-858-784-9090. E-mail address: cpotter@scripps.edu (C.S. Potter). 1047-8477/$ - see front matter Ó 2003 Elsevier Inc. All rights reserved. doi:10.1016/j.jsb.2003.09.033 Journal of Structural Biology 145 (2004) 3–14 Journal of Structural Biology www.elsevier.com/locate/yjsbi