Selective Attention to Auditory Stimuli: A Brain-Computer Interface Paradigm N. Jeremy Hill, 1 Thomas Navin Lal, 1 Michael Schr¨ oder, 2 Thilo Hinterberger, 2 Niels Birbaumer 2 and Bernhard Sch¨ olkopf 1 1 Max Planck Institute for Biological Cybernetics, T¨ ubingen 2 Eberhard Karls University, T¨ ubingen Introduction The aim of research into brain-computer interfaces (BCI) is to allow a person to control a computer using signals from the brain, without the need for any muscular movement—for example, to allow a totally para- lyzed patient to communicate. We report the results of an experiment on normal subjects, designed to develop a paradigm in which a user can make a binary choice. Other researchers have employed paradigms in which users imagine moving either their left or right hand [1], or in which they are trained over several weeks to control slow cortical potentials [2]. Here, we attempt to classify EEG signals that occur in response to two simultaneous auditory stimulus streams. To communicate a binary de- cision, the subject focuses attention on one of the two streams, left or right. Experiment 1 0 0.5 1 1.5 2 2.5 3 3.5 4 1 0.5 0 0.5 1 time (sec) amplitude monitor deviant tones (quieter, different frequency) one trial: 8 x 50-msec pure-tone beeps from left speaker, 7 from right Stimulus parameters left stimulus right stimulus sequence sequence frequency of standard beeps 1500 Hz 800 Hz frequency of deviant beeps 1560 Hz 840 Hz period of sequence 490 msec 555 msec SOA of first beep 0 msec 70 msec Trial structure 0.0 1.0 1.5 6.0 6.5 2.0 sec fixation point on screen visual cue auditory stimulus EEG recording + 2-4 sec pause • While the fixation point was on, subjects were asked to maintain fixation and remain motionless. • The visual cue was an arrow pointing left or right, indicating which of the two stimulus streams the subject should attend to. • To help them focus their attention on one side, subjects were asked to count the deviant beats in the attended sequence, and ensure that they could “play back” the attended sequence in their mind when the trial had finished. The number of deviant beats per sequence was either 1 (on 67% of trials) or 2 (33% probability) was randomized. • Four subjects each performed 8 blocks, each consisting of 50 trials. EEG signals were recorded from 39 silver chloride electrodes at a sampling rate of 256Hz, band-pass filtered between 0.1Hz and 40Hz. Selective attention to auditory stimuli is known to have a measurable effect on EEG signals. Hillyard et al. [3] and others reported in the 60’s and 70’s that selective attention in a dichotic listening task caused a measurable modulation of EEG signals (see [4,5] for a review). This was significant when signals were averaged over a large number of instances, but our aim was to dis- cover whether single trials are classifiable with a high enough accuracy to be useful in a BCI. Could this effect be used for BCI? To estimate single-trial classification accuracy, we trained a linear Sup- port Vector Machine [6] on 90% of the data before testing it on the remaining 10%. This was carried out ten times, taking a different 10% each time (ten-fold cross-validation). Accuracy is estimated as the aver- age rate at which the trained SVM correctly predicted, for the ten 10% test subsets, whether left or right had been the focus of attention. 50% corresponds to chance performance (no generalization possible). Perfor- mance varied according to the preprocessing method used to represent the EEG signals—the following four methods were used to determine the features of each EEG channel to be given as input to the classifier: (A) 266 samples of the raw EEG signal, obtained by averaging the re- sponses in the 490 msec intervals following each beat of the left signal, and concatenating these with the average responses in the 550 msec intervals following each beat of the right signal; (B) as for (A), except that the average difference between responses fol- lowing deviant stimuli and standard stimuli was taken for each trial; (C) 81 coefficients of the amplitude spectrum of the whole-trial signal, representing harmonics from 5–25 Hz; (D) 3 coefficients of an auto-regressive model fitted to the EEG trace from the whole trial duration. Their relative performance is shown in the following table (× represents an accuracy not significantly different from chance): Accuracy for each of 4 subjects prep. type AG CR HM JH (A) × × × × (B) × × × × (C) 58% ± 3 61% ± 2 64% ± 2 64% ± 2 (D) 56% ± 2 63% ± 3 62% ± 2 58% ± 3 Yes (although not very accurately). . . Preprocessing types (A) and (B), in which other investigators have found significant differences between attend-left and attend-right trials when the average is taken across many trials, are clearly not classifiable on a single-trial basis (at least for these stimuli—however, see below). Preprocessing types (C) and (D) demonstrated that this paradigm can produce signals that are classifiable at significantly higher than chance performance. However, accuracy is still very poor (corresponding to a communication capacity of between 0.02 and 0.06 bits per trial). The Recursive Channel Elimination method of Lal et al. [7] (see also Schr¨ oder et al, TWK2004 p.50) assesses the effect on the SVM deci- sion boundary of removing a set of features. By removing the least influential EEG channels recursively, we obtain the following curves for preprocessing type (C) and (D): 0 10 20 30 40 40 45 50 55 60 65 70 accuracy (%) number of EEG channels used FFT preprocessing (C) 0 10 20 30 40 40 45 50 55 60 65 70 accuracy (%) number of EEG channels used A-R preprocessing (D) JH HM CR AG JH HM CR AG Note that preprocessing (D) has already reduced the number of dimen- sions in the data drastically before EEG channels are selected out. It is not surprising, therefore, that it is less robust to channel elimination than (C). Preprocessing type (C), taken over all subjects, is the most successful preprocessing, and the channel elimination results indicate the dimensionality of the data can be reduced by about a half before input into the SVM, or to put it another way, that only half of the EEG channels need be used to achieve roughly the same performance. Removal does not improve our accuracy, however, and the electrode positions of the eliminated channels showed no particular spatial pattern: the information in the signals is was found to be fairly evenly distributed over the scalp. . . . but there is room for improvement. Experiment 2 (preliminary results) The stimuli in Experiment 1 were designed to replicate as closely as pos- sible the conditions used by Hillyard et al. [3]: the frequencies, durations and periods of repetition of the beeps were the same or comparable. Pre- liminary results from one subject in a different condition suggest that the stimuli may be adapted to yield improved results in the same paradigm. In this Experiment 2, we used broad-spectrum sounds chosen to be as easily distinguishable from one another as possible: a “hi-hat” cymbal and a bass drum from a midi drum sound-set were used as the two standard stimuli, repeating six times each with periods of 680 msec and 550 msec respectively. In this experiment, “deviant” beats were silent beats: the fourth or fifth beat was simply missing from the sequence on two thirds of the trials (the remaining third had no missing beats). prep. type Accuracy (A) 65% ± 2 (B) 67% ± 4 (C) 61% ± 3 (D) 59% ± 3 In the EEG literature, the effect of attention is often seen most clearly in the contrast between responses to deviant and those to standard stimuli. This may be reflected in the fact that although preprocessing types (A) and (B) seem to have the same initial level of accuracy, (B) is perhaps slightly more stable under recursive elimination of channels (even though, in this experiment, one third fewer trials were available to this preprocessing). 0 10 20 30 40 40 45 50 55 60 65 70 accuracy (%) number of EEG channels used preprocessing (B) preprocessing (A) Summary We have shown that, in principle, selective attention to auditory stimu- lus streams can provide differences in EEG signals that are classifiable on a single-trial basis for the purposes of BCI. Though error rates are currently high in comparison with imagined-movement or slow-cortical- potential approaches to BCI, there is reason to believe that performance may in future be improved by appropriate optimization of the stimulus. References [1 ] Pfurtscheller, G. et al. (1998) Separability of EEG Signals Recorded during Right and Left Motor Imagery using Adaptive Autoregressive Parameters. IEEE Transactions on Rehabilitation Engineering, 6(3):316–325. [2 ] Birbaumer, N. et al. (2000): The Thought Translation Device (TTD) for Completely Paralyzed Patients. IEEE Transactions on Rehabilitation Engineering, 8(2):190–193. [3 ] Hillyard, S. et al. (1973): Electrical Signs of Selective Attention in the Human Brain. Science 182:177–80. [4 ] N¨ a¨ at¨ anen, R. (1982): Processing Negativity: an Evoked-Potential Reflection of Selective Attention. Psychological Bulletin 92(3):605– 640. [5 ] N¨ a¨ at¨ anen, R. (1990): .The Role of Attention in Auditory Infor- mation Processing as Revealed by Event-Related Potentials and other Brain Measures of Cognitive Function Behavioral and Brain Sciences 13:201–288. [6 ] Sch¨ olkopf, B. and Smola, A. (2002): Learning with Kernels. MIT Press Cambridge. [7 ] Lal, T. N. et al. (2003): Support Vector Channel Selection in BCI. MPI Technical Report 120, Max Planck Institute for Biological Cybernetics, T¨ ubingen.