IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 52, NO. 6, JUNE 2005 1065 Quantifying Motion in Video Recordings of Neonatal Seizures by Robust Motion Trackers Based on Block Motion Models Nicolaos B. Karayiannis*, Senior Member, IEEE, Yaohua Xiong, James D. Frost, Jr., Merrill S. Wise, and Eli M. Mizrahi Abstract—This paper introduces a methodology for the develop- ment of robust motion trackers for video based on block motion models. According to this methodology, the motion of a site be- tween two successive frames is estimated by minimizing an error function defined in terms of the intensities at these frames. The pro- posed methodology is used to develop robust motion trackers that rely on fractional block motion models. The motion trackers de- veloped in this paper are utilized to extract motor activity signals from video recordings of neonatal seizures. The experimental re- sults reveal that the proposed motion trackers are more accurate and reliable than existing motion tracking methods relying on pure translation and affine block motion models. Index Terms—Affine motion model, block motion model, frac- tional motion model, generalized fractional motion model, motion tracking, motor activity signal, neonatal seizure, robust motion tracking, translation motion model, video recording. I. INTRODUCTION B LOCK motion models have been a popular tool in many applications involving motion tracking, which include motion-compensated video coding [8], [16], and video-based surveillance [26]. A novel application of motion tracking is the extraction of quantitative motion information from video recordings of neonatal seizures [9]–[15]. Seizure occurrence is one of the most frequent clinical signs of central nervous system dysfunction in the newborn [3], [17], [30], and seizures have been associated with significant long-term sequelae such as neurological impairment, developmental delay, and post- natal epilepsy. The prompt identification of clinical seizures when they occur in the newborn, the subsequent determination of their etiology, and the institution of specific therapy may significantly reduce associated morbidity. Manuscript received April 27, 2004; revised October 31, 2004. This work was supported by the National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health under Grant 1 R01 EB00183, and by the National Institute of Neurological Disorders and Stroke under Contract N01-NS-2316. Asterisk indicates corresponding author. *N. B. Karayiannis is with the Department of Electrical and Computer En- gineering, N308 Engineering Building 1, 4800 Calhoun Road, University of Houston, Houston, TX 77204-4005 USA (e-mail: karayiannis@uh.edu). Y. Xiong is with the Department of Electrical and Computer Engineering, University of Houston, Houston, TX 77204-4005 USA. J. D. Frost, Jr. is with the Peter Kellaway Section of Neurophysiology, De- partment of Neurology, Baylor College of Medicine, Houston, TX 77030 USA. M. S. Wise, and E. M. Mizrahi are with the Peter Kellaway Section of Neu- rophysiology, Department of Neurology, Baylor College of Medicine, Houston, TX 77030 USA and also with the Section of Pediatric Neurology, Department of Pediatrics, Baylor College of Medicine, Houston, TX, 77030 USA. Digital Object Identifier 10.1109/TBME.2005.846715 The development of portable EEG/video/polygraphic moni- toring techniques has allowed investigators to assess and char- acterize neonatal seizures at the bedside and has permitted retro- spective review [1], [3], [7], [19], [21], [22]. Quantitative video analysis may supplement and extend human analysis and may generate novel methods for extracting relevant information from paroxysmal neonatal behaviors. Such refined analysis may shed light on specific motor activity patterns or attributes that con- stitute true seizures, as compared to repetitive behaviors that do not represent seizures and, consequently, do not have the same clinical relevance. This could also uncover key motor signatures that are not recognized using traditional visual analysis or mon- itoring of body/limb motion by EMG or accelerometry [4]. Neonatal seizures can be quantified in terms of temporal motion strength and motor activity signals [9], [12]. Tem- poral motor activity signals can be extracted from video by projecting to the horizontal and vertical axes an anatomical site located on a moving body part that may be affected by a seizure. The extraction of motor activity signals requires an automated procedure capable of tracking a certain anatomical site located on a moving body part throughout the frame sequence. Motor activity signals were extracted in an earlier study [12] by the KLT algorithm [25], [28]. The KLT algorithm was generally successful in extracting motor activity signals from video recordings of neonatal seizures. However, in some cases the algorithm lost anatomical sites that were located on moving body parts tracked throughout the frame sequence [9], [12]. The susceptibility of the KLT algorithm to “lost sites” motivated alternative approaches aimed at the development of more accurate and reliable motion trackers. Motion tracking was performed in this study by employing adaptive block matching to track a block of pixels located on a moving body part throughout a sequence of frames [11]. The development of more reliable and accurate motion trackers focused on several aspects of motion tracking based on block motion models, including the treatment of the minimization problem [13], [15] and the use of more sophisticated block motion models [10], [14], [25], [29]. More specifically, these approaches involved a rigid motion model that allows for rotation and uniform scaling [10] and a deformable motion model that incorporates an affine transformation in addition to translation [14], [25], [29]. This paper introduces a methodology for the development of robust motion trackers for video. The proposed motion trackers rely on fractional block motion models, which can be seen as the essential generalization of the affine block motion 0018-9294/$20.00 © 2005 IEEE