Detection of Cell Division Time and Number of Cell for in Vitro Fertilized (IVF) Embryos in Time-Lapse Videos with Deep Learning Techniques Hüseyin KUTLU #1 , Engin AVCI *2 # Besni Vocational School, Department of Computer Use, Adıyaman University Adıyaman University, Adıyaman / TURKEY 1 hkutlu@adiyaman.edu.tr * Faculty of Technology, Software Engineering Department, Fırat University Fırat University, Elazığ / TURKEY 2 enginavci@firat.edu.tr Abstract—Embryo development is one of the key factors that provide pregnancy in IVF treatments. The healthy development of the embryo directly affects the realization of the pregnancy. Being able to monitor the development of the embryo increases the rate of conception. It is expected that the embryo will reach a structure of 2-4-8 cells at a specific time with mitosis from a cell. Early or late proliferation from this time indicates that the embryo is unhealthy. In this study, embryo cells were detected with deep learning-based object detector that is Faster Region- based Convolutional Neural Network (Faster R-CNN). ID numbers have given to the detected cells and cells were tracked. With a data structure, the cells have taken an id number. The position change estimations of the cells were performed with Kalman Filter. Hungarian algorithm was used to correlate cells in video frame changes. Cell tracking was performed with the proposed method and division times and cell counts were obtained from time-lapse videos. The Faster R-CNN is trained and tested with Mouse Embryo Tracking Database which is a public embryo database. Faster Region-based Convolutional Neural Networks (Faster R-CNN) have recently emerged as superior for many image detection tasks. In this study, it has been shown that using Faster R-CNN object detector to cell tracking up to 4 cells can achieve competitive results. Keywords—Cell Tracking, Cell Counting, Cell Division, Faster R-CNN, Embryo Cell Detection I. INTRODUCTION The important problem in IVF treatments is to determine which embryo will occur during pregnancy [1]. After fertilization of eggs, embryos continue to develop in incubators. In incubators temperature, oxygen and carbon dioxide amount is stabile for embryos develop. Embryos are examined by embryologists 2-3 times until the day of transfer and embryos considered to be the best are transferred. As these examinations are conducted outside, they may disrupt embryo quality. It is preferred that these examinations be carried out in the incubator, which is similar to the natural environment of the embryo. It is called zygote before it begins to divide into a newly fertilized egg. After the first 24 hours (day 1), the zygote is divided into two cells and becomes embryos. On day 2, the embryo becomes 4-cell (blastomer) and the third day should have 6-9 cells. The development of the embryo to this point is controlled by the mother's genes. When the 8-cell stage is reached, the embryonic genome begins to control its development. On day 4, the embryo has 16 to 32 cells [2]. At this point the embryo is called 'morula'. All cells of the embryo are identical until the Morula stage. As described above, the development of the embryo takes days. During this time, the embryo can be examined with a time-lapse camera. The image taken from time-lapse camera can be examined by image processing techniques. The number of embryo cells and the dividing time can be obtained from an image processing software. For human in vitro fertilized (IVF) embryos, cell division timing has been shown to correlate with embryonic viability [3], [4], [5], [6]. In recent years, a number of studies related to time-lapse imaging of early embryos have been published. In [8] Arsa et al. have proposed a method to predict the number of blastameres of the embryo time-lapse using Conditional Random Field (CRF) based on Bag of Visual Words (BoVW). In [9] Khan et al. were segmented the embryos by traditional methods and [10] In [10] their proposed method is based on a linear chain Markov model that estimates the number and location of cells at each time step. In this study, embryo cells were detected with Faster R- CNN. With a data structure, the cells have taken an id number. The position change estimations of the cells were performed with kalman filter. Hungarian algorithm was used to correlate cells in video frame changes. Cell tracking was performed with the proposed method and division times and cell counts were obtained from time-lapse videos. The limitation of the tracking-by-detection methods is that the tracking performance depends heavily on detection results. Faster R-CNN is a deep learning object detector algorithm with high accuracy rate [7]. The foundation of FR-CNN is the CNN architecture. In this study, different CNN architectures were compared for the mentioned embryo cell tracking problem. The rest of this study is organize as follows. In Section 2, the method proposed is mentioned briefly. Experimental