Deep Learnt Random Forests for Segmentation of Retinal Layers in Optical Coherence Tomography Images Sri Phani Krishna Karri, Debdoot Sheet, Arpan Guha Mazumder, Sambuddha Ghosh, Debjani Chakraborty, Jyotirmoy Chatterjee and Ajoy K. Ray Abstract— Retina is a crucial segment of the eye and is composed of photoreceptors comprising the retinal pigment epithelium (RPE). Age-related macula degeneration (AMD) is a condition in which deposition of drussen material between the RPE and Bruch’s membrane triggers an irreversible visual loss and ophthalmologists rely on optical coherence tomography (OCT) for examining the variation in width of anterior coat and the RPE located below it. Since a high degree of stratified contrast is not observed between these layers, automated seg- mentation methods are deployed to reduce reporting variability. Here we propose a deep learning technique for retinal layer segmentation in OCT image sequences. This approach uniquely facilitates learning to extract features characteristic of different layers. It is experimentally validated using images of 20 AMD + 20 healthy subjects, and 100 image per subject. Anterior coat is segmented with 0.97, RPE with 0.92, posterior coat with 0.99 accuracy. I. I NTRODUCTION Vision loss in the elderly on account of age related macula degeneration (AMD) is caused by deposition of drussen material between the retinal pigment epithelium (RPE) and the posterior coat starting at Bruch’s membrane. Pptical coherence tomography (OCT) is used for clinically examining the differential change in thickness of the anterior coat and the RPE for early diagnosis of AMD [1]. Most retinal layers have characteristic cellular composition and offer distinct stratified contrast, that is not strongly observed between RPE and and posterior coat originating at Bruch’s membrane [1]. Retinal later segmentation approaches have been proposed [2] to overcome this limitation and their performance varies based on (a) nature of feature descriptors used as well as (b) the learning methodology. In order to reduce segmentation variability we propose a deep learning approach [3] for retinal layer segmentation. II. MATERIALS AND METHODS Retinal OCT images of 20 AMD + 20 healthy subjects, and 100 image per subject from [1] are used for experiments. S. P. K. Karri, D. Sheet, A. Guha Mazumder and J. Chatterjee are with the School of Medical Science and Technology, Indian Institute of Technology Kharagpur, Kharagpur, WB, India. (email: pkkarri.mm@iitkgp.ac.in) D. Chakraborty is with the Department of Mathematics, Indian Institute of Technology Kharagpur, Kharagpur, WB, India. S. Ghosh is with the NBMCH, Darjeeling, WB, India. A. K. Ray is with the Dept. of Electronics and Electrical Comm. Engg., Indian Institute of Technology Kharagpur, Kharagpur, WB, India. Fig. 1. The anterior coat, RPE and posterior coat segmented in retinal OCT with deep learnt random forest. The abrupt nature of RPE thickness characteristic of AMD is clearly visible in segmented results despite the low contrast between these layers in grayscale OCT images. Fixed width overlapping patches from the grayscale OCT image are used initially for training a stack of two auto- encoders (AE) in a greedy manner. The AE parameters are fine tuned using labelled training data through logistic re- gression (LR) and error back-propagation. We consider four classes, viz., anterior coat, RPE and posterior coat; and the non-tissue region. Details of this architecture can be found in [3]. Since, LR is characteristic to learn patterns efficiently when they are separable in a logarithmic transformed space. In absence of this property in the data, we can learn patterns from the AEs using a random forest [4] when patterns are embedded in a higher order manifold. III. RESULTS AND CONCLUSION Experimental evaluation as presented in Fig. 1 shows accuracy of 0.97 in segmenting anterior coat, 0.92 in RPE and 0.99 in posterior coat, and substantiates efficacy of retinal layer segmentation in OCT images using deep learnt random forests. REFERENCES [1] S. Farsiu, S. J. Chiu, R. V. O’Connell, F. A. Folgar, E. Yuan, J. A. Izatt, and C. A. Toth, “Quantitative classification of eyes with and without intermediate age-related macular degeneration using optical coherence tomography,” Ophthalmol., 2013. [2] R. Kafieh, H. Rabbani, M. D. Abramoff, and M. Sonka, “Intra-retinal layer segmentation of 3d optical coherence tomography using coarse grained diffusion map,” Medical Image Analysis, vol. 17, no. 8, pp. 907–928, Dec. 2013. [3] D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, and S. Bengio, “Why does unsupervised pre-training help deep learning?,” J. Mach. Learn. Res., vol. 11, pp. 625–660, 2010. [4] A. Criminisi, J. Shotton, and E. Konukoglu, “Decision forests: A unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning,” Found. Trends. Comput. Graph. Vis., vol. 7, no. 3, pp. 81–227, Feb. 2012.