Dynamic Time Warping: A Single Dry Electrode EEG Study in a Self-paced Learning Task Takashi Yamauchi, Kunchen Xiao, Casady Bowman Department of Psychology Texas A&M University College Station, Texas, USA takashi-yamauchi@tamu.edu Abudullah Mueen Department of Computer Science University of New Mexico Albuquerque, New Mexico, USA mueen@cs.unm.edu Abstract—This study investigates dynamic time warping (DTW) as a possible analysis method for EEG-based affective computing in a self-paced learning task in which inter- and intra- personal differences are large. In one experiment, participants (N=200) carried out an implicit category learning task where their frontal EEG signals were collected throughout the experiment. Using DTW, we measured the dissimilarity distances of EEG signals between participants and examined the extent to which a k-Nearest Neighbors algorithm could predict self-rated feelings of a participant from signals taken from other participants (between-participants prediction). Results showed that DTW provides potentially useful characteristics for EEG data analysis in a heterogeneous setting. In particular, theory- based segmentation of time-series data were particularly useful for DTW analysis while smoothing and standardization were detrimental when applied in a self-paced learning task. Keywords—DTW; Self-paced learning; Neurosky MindWave I. INTRODUCTION When Calvo and D’Mello [1] reviewed the state of the art in affective computing research in 2010, emotion analysis employing electroencephalography (EEG) was relegated to the background: “Regrettably the cost, time resolution, and complexity of setting up experimental protocols that resemble real-world activities are still problematic issues that hinder the development of practical applications that utilize these techniques” (p. 26) [1]. Notable advances have been made in EEG-based analysis since Calvo and D’Mello’s review primarily due to pioneering work by [2-8]; however, the same problems still plague EEG- based emotion analysis. Unlike face-based or voice-based methods, collecting EEG signals in a natural setting is far more complicated. Because many useful EEG features are time- locked, stimulus presentation and response collection need to be tightly controlled. A typical experiment consists of a brief presentation of emotional stimuli (e.g., affective images taken from the IAPS [9]) where participants’ EEG signals are collected in an event-locked brief time-window [3]. Because almost all EEG-based studies employ a within-subjects design, one can come up with viable EEG features, but these features are barely useful for emotion recognition beyond the same person in the same task [10]. What analysis procedure can be applied in a self-paced learning environment (e.g., intelligent tutoring systems) where individual users interact with computers freely for a long period (e.g., more than 10 minutes)? In an attempt to investigate this questions, we employed dynamic time warping [11] as a possible analysis method and examined its applicability in a self-paced naturalistic learning environment where EEG signals were collected continuously throughout the experiment by a single dry electrode mobile device (Neurosky Mindwave Mobile). II. RELATED WORK A. EEG-based affective computing Complex human behavior, such as perception, object recognition, motor coordination, attention, and emotion expression and experience, results from synchronous coordination between excitatory and inhibitory neurons in cortical and subcortical areas of the brain [12]. For example, to reach and grab an object by a hand, excitatory signals sent from the primary motor cortex are modulated by inhibitory signals from the basal ganglia to control thousands of muscle fibers [13]. Likewise, in experiencing and expressing affects and motivation, signals processed in the limbic system are modulated by signals sent from the prefrontal cortex [14-16], generating a variety of oscillatory interactions [12, 17]. These excitatory and inhibitory neural interactions license complex nonlinear human behavior, giving rise to oscillatory electrical signals on the scalp. As such, affective states, such as temperament, personality and cognitive dissonance, are known to be associated with EEG signals that are captured in frontal lobe regions [18, 19]. Despite its compelling physiological basis, EEG-based affective computing has not fully materialized due to the following limitations: its data acquisition cost is prohibitively high as compared to other affective computing methods using facial expressions, voice, gaits, and cursor motion; EEG-based studies require strict experimental settings where the presentation of stimuli should be well marked and time-locked. Due to these limitations, existing EEG studies in affective computing are mostly conducted with a small number of subjects (Ns < 20) in a restricted setting where emotionally salient stimuli such as pictures, film or music clips, are shown 978-1-4799-9953-8/15/$31.00 ©2015 IEEE 56 2015 International Conference on Affective Computing and Intelligent Interaction (ACII)