Video-based Activity and Movement Pattern Analysis in Overnight Sleep Studies Wen-Hung Liao Chien-Ming Yang Department of Computer Science Research Center for Mind, Brain and Learning Department of Psychology National Chengchi University, Taipei, TAIWAN {whliao,yangcm}@nccu.edu.tw Abstract We present a non-contact monitoring system to measure the quality of sleep using near-infrared video in this paper. We envision a smart home environment in which a processing module can be installed in the bedroom to record and monitor sleep in a non- invasive manner. We describe the procedure adopted to infer motion information and discuss the method for estimating wake/sleep status from the acquired video. Performance of the proposed system was evaluated through comparison with simultaneous recordings of actigraph and polysomnography (PSG) data. 1. Introduction One third of our life is spent in sleeping. The other two-thirds are also affected by the quality of sleep during the previous night. Empirical studies have documented that poor sleep has negative impacts on daytime mood, cognitive performance and overall life quality. Good sleep is therefore essential for maintaining a quality life. In most laboratories, polysomnography (PSG) is utilized to study sleep-related disorders. Standard overnight PSG, however, included continuous recordings of several channels of physiological signals such as EEG, EOG, EMG and EKG. Not only is it time-consuming to attach the sensors, the high cost also prohibits the deployment of such devices in a home environment. Our objective is to develop a system that can monitor sleep in a real-life scenario rather than laboratory settings. As such, the system should be capable of measuring sleep-related signals with low cost, non-tethered sensors. Previous studies have shown good validity to estimate sleep/wake status with activity monitors. Also, individuals in different sleep status were shown to have different breathing patterns and movement profiles. Accordingly, we aim to construct a video-based system that integrates information acquired from near-infrared image sequences, including video monitors to detect movements and postures, and activity monitors to estimate sleep status. The efficacy of the proposed approach will be established by comparing the data obtained from standard PSG measurements and wrist actigraph recordings. The rest of this paper is organized as follows. In section 2 we discuss related video-based sleep monitoring systems and the role of video in traditional PSG. We then identify several issues in the processing of near-infrared video. Section 3 explains the procedure of extracting activity record as well as movement patterns from the video. Section 4 briefly summarizes the basic operating principles of acti- watches and the rule employed to infer wake/sleep status from activity logs. Section 5 compares the results of PSG measurements, wrist actigraph recordings and the proposed video-based monitoring system. Section 6 concludes this paper with a brief conclusion and outlook on future improvements. 2. Video-based sleep monitoring systems Traditionally, video is employed in sleep labs as an auxiliary channel for the technician or assistant to monitor the subject’s status during the experiment. Video is usually captured at a very low resolution (160x120) to conserve disk space since the recording 978-1-4244-2175-6/08/$25.00 ©2008 IEEE