Real-Time 3-D Sensing, Visualization and Recognition of Dynamic Biological Microorganisms SEOKWON YEOM, INKYU MOON, AND BAHRAM JAVIDI, FELLOW, IEEE Invited Paper We introduce optical imaging techniques for three-dimen- sional (3-D) visualization and identification of microorganisms. Three-dimensional sensing and reconstruction is performed by single-exposure on-line (SEOL) digital holography. A coherent microscope-based Mach–Zehnder interferometer records Fresnel digital holograms of microorganisms. Complex amplitude holo- graphic images are computationally reconstructed at different depths by an inverse Fresnel transformation. For pattern recog- nition/identification, two approaches are addressed. One is 3-D morphology-based recognition and the other is shape-tolerant 3-D recognition. In the first approach, a series of image recog- nition techniques is used to analyze 3-D geometrical shapes of microorganisms, which is composed of magnitude and phase distributions. Segmentation, feature extraction, graph matching, feature selection, training, and decision rules are presented. For the second approach, a number of sampling segments are arbitrarily extracted from the reconstructed 3-D biological microorganism. These samples are processed using a number of cost functions and statistical inference theory for the equality of means and equality of variances between the sampling segments. Experimental results with sphacelaria alga, tribonema aequale alga, and polysiphonia alga are presented. Keywords—Automatic optical inspection, biological imaging, feature extraction, holographic interferometry, image object detec- tion, image object recognition, image pattern recognition, image reconstruction, image segmentation, three-dimensional (3-D) optical imaging. I. INTRODUCTION Three-dimensional (3-D) optical and image processing for pattern recognition have been developed as real-time and au- Manuscript received July 31, 2005; revised November 1, 2005. This work was supported by the Defense Advanced Research Projects Agency (DARPA), U.S. Department of Defense (DOD). The authors are with the Electrical and Computer Engineering De- partment, University of Connecticut, Storrs, CT 06269 USA (e-mail: yeom@engr.uconn.edu; inkyu.moon@huskymail.uconn.edu; bahram@ engr.uconn.edu). Digital Object Identifier 10.1109/JPROC.2006.870691 tomated approaches for 3-D survey and inspection. Two-di- mensional (2-D) and 3-D optoelectronic image processing with pattern recognition have been studied to identify spe- cific objects in unknown scenes [1]–[27]. Increased interests in information processing with 3-D imaging systems are well reflected in recent contexts [11], [12]. For example, digital holography has been utilized for object recognition as well as image encryption [13]–[25]. Applications in 3-D integral imaging have been extended to automatic target recognition (ATR) as well as depth estimation [26], [27]. One can find various military and civilian/industrial applications of automated recognition systems of microor- ganisms, including biological weapon detection in security and defense; diagnosis of diseases, food safety investigation, and medical and health care; and ecological monitoring, quantitative analysis of microorganisms in wastewater treat- ment or oceanography, etc. However, real-time automated recognition of microscopic biological objects in dynamic scenes is very challenging. The interclass diversity of microorganisms in size and shape is comparable to the intraclass diversity. Their geometrical shape is relatively simple and changing by the motility and growth, which are influenced by external factors [28]. A mi- croorganism can appear as an individual object or form a group or clutter with arbitrary complexity. Therefore, spe- cial care with the morphological and biological characteris- tics of microorganisms should be considered to enhance the discrimination capability of the recognition system. Most research and development in this field has been performed to recognize specific features of microorganisms based on captured 2-D intensity images [29]–[33]. The recognition and identification of tuberculosis bacteria [29] and vibrio cholera [30] have been studied based on their colors and 2-D shapes. In [31], bacteria in a wastewater treatment plant are identified by morphological descriptors. The aggregation of streptomyces is classified into different phases by measuring the aggregation size and reaction 0018-9219/$20.00 © 2006 IEEE 550 PROCEEDINGS OF THE IEEE, VOL. 94, NO. 3, MARCH 2006