A MULTIPLE COVARIANCE APPROACH FOR CELL DETECTION OF GRAM-STAINED SMEARS IMAGES Matthew Crossman ⋆ , Arnold Wiliem ⋆ , Paul Finucane † , Anthony Jennings † , and Brian C. Lovell ⋆ ⋆ The University of Queensland, Australia † Sullivan Nicolaides Pathology, Australia ABSTRACT Microscope examination of Gram stained clinical specimens is used for aiding the diagnosis of patients with infectious diseases. In high volume pathology laboratories, this man- ual microscopy examination is considered time consuming and labour intensive. Unfortunately, despite the great bene- fits offered from the application of Computer Aided Diagno- sis (CAD) systems, to our knowledge, the highest automa- tion stage for Gram stained slide analysis is only at the pre- analytical process. This paper takes the first steps towards the application of computer vision to direct smear, Gram stained images. To that end, we present a novel Gram stain image dataset. In addition, we also propose a multiple covariance approach for leukocyte and epithelial cell detection in Gram stain images. Each covariance matrix represents a particu- lar image region characterising the cell’s deformed structure. As covariance matrices form points on an Symmetric Posi- tive Definite (SPD) manifold, the traditional Euclidean-based analysis cannot be used. As such, we first map the manifold points into the Reproducing Kernel Hilbert Space (RKHS). The analysis is done via a novel kernel similarity function that allows comparison between sets of covariance matrices. The proposed approach is contrasted, in the proposed dataset, with two recent state of the art methods in pedestrian detec- tion: Histogram Of Gradient (HOG) and the traditional single covariance matrix approach. We found that the proposed ap- proach outperformed both of these methods. Index Terms— Gram stain analysis, direct smears, Cell detection, Riemannian manifolds, Symmetric Positive Defi- nite Matrix group 1. INTRODUCTION Recently there has been growing interest in applying image analysis to pathology test images [1, 2, 3, 4]. More precisely, Computer Aided Diagnosis (CAD) systems were developed to automatically provide analysis based on the input images. Results produced by these methods can be used to support the scientists’ manual/subjective analysis; making the test results more reliable and consistent. In microbiology, it is known that microscopic examination of Gram stained preparations This research was funded by Sullivan Nicolaides Pathology (SNP), Aus- tralia and the Australian Research Council (ARC) Linkage Projects Grant LP130100230. (Leukocytes detection) (Epithelial detection) (Epithelial detection) (Non-cell example) Fig. 1. Some detection bounding box results from our pro- posed approach. Note that the cells of interest such as leuko- cytes and epithelial cells have extreme variability in shapes. In some cases, some non-cell objects also have similar appear- ance to these cells. Our proposed approach is able to address these variabilities. of clinical specimens is valuable for physicians who are man- aging patients with infectious diseases [5]. This is due to the fact that it is a rapid and cost effective procedure. The Gram staining protocol is a staining procedure ap- plied to tissue samples on glass slides, which facilitates classi- fication of cells under bright field microscopes as either Gram positive or Gram negative [6]. This classification aids the sci- entist in estimating the abundance of several cell classes in the smear. The relative population of white blood cells and bacteria cells, together with epithelial cells gives an efficient first diagnosis of bacterial infections. For pathology laboratories handling large volumes of patient samples, the manual examination of each slide sig- nificantly consumes valuable time of scientists. These issues could be addressed using CAD systems designed for cell recognition with the detection statistics used for population