ISSN 1054-6618, Pattern Recognition and Image Analysis, 2017, Vol. 27, No. 3, pp. 599–609. © Pleiades Publishing, Ltd., 2017. A New Method for Automating the Investigation of Stem Cell Populations Based on the Analysis of the Integral Optical Flow of a Video Sequence O. V. Nedzvedz a, *, S. V. Ablameyko b, **, I. B. Gurevich c, ***, and V. V. Yashina c, **** a Belarusian State Medical University, pr. Dzerzhinskogo 83, Minsk, Belarus; United Institute of Informatics Problems, National Academy of Sciences of Belarus, ul. Surganova 6, Minsk, Belarus b Belarusian State University, pr. Nezavisimosti 4, Minsk, Belarus c Federal Research Center “Computer Science and Control,” Russian Academy of Sciences, 44, Building 2, Vavilov str. Moscow, 119333, the Russian Federation *e-mail: Olga_Nedzved@tut.by **e-mail: ablameyko@bsu.by ***e-mail: igourevi@ccas.ru ****e-mail: werayashina@gmail.com Abstract—A new method is presented for automating the investigation of stem cell populations based on the automatic analysis of video sequences of images. For automatic image analysis, an integral optical flow appa- ratus is used. The proposed method classifies dynamic objects by constructing a pyramid of the integral opti- cal flow. The main stages of the method are as follows: image capturing and processing, segmentation, mea- surement, and tissue description generation. Experimental tests confirm that the proposed method is capable of identifying the main stages of cellular development (mitosis, differentiation, and apoptosis). Keywords: image analysis, automation of scientific research, optical flow, stem cells, and cell population dynamics DOI: 10.1134/S1054661817030221 1. INTRODUCTION The potential of using mathematical and com- puter-based methods for automation of biomedical research, particularly in neurosciences and brain research, as well as for interpretation of their results, is as extensive as the scope of these fields themselves. It is commonly accepted that in natural sciences, the role of mathematical and information processing methods is limited to [2, 3, 5] the following tasks: 1. formal set-up of a problem; 2. automated processing, analysis, and interpreta- tion of experimental results; 3. construction of mathematical and simulation models of objects and information, biological, physio- logical, physical, chemical, and other material and energy processes; 4. computer experiments with mathematical and simulation models; 5. support of intelligent decision making based on the analysis of research and simulation results; 6. extraction of knowledge from experimental data and models, as well as its structuring and formal description. In biology and medicine, mathematical and infor- mational investigations are focused on [2, 3, 5] (1) automated extraction of information and knowledge from experimental data; (2) modeling at the levels of individual neurons, neural networks, and brain sections. For these purposes, it is required to (1) develop methods and software for analysis and modeling; (2) develop and investigate, both theoretically and experimentally, models of the nervous system and processes occurring therein; (3) develop methods, tools, databases, and knowl- edge bases for neurosciences at all levels of analysis of the vital activity mechanisms and functions of the human body. Mathematical and computer-science approaches are now widely used in biomedicine, particularly in neurosciences [2–5]. An important direction in disease diagnostics involves storing, processing, and analyzing the data extracted from digital images. An image is one of the APPLIED PROBLEMS Received April 18, 2017