USING RECURRENT NEURAL NETWORKS FOR P300-BASED BRAIN-COMPUTER INTERFACES O. Tal, D. Friedman The Advanced Reality Lab, The Interdisciplinary Center, Herzliya, Israel Contact: doronf@idc.ac.il ABSTRACT: P300-based spellers are one of the main methods for electroencephalogram (EEG)-based brain- computer interface, and the detection of the target event with high accuracy is an important prerequisite. The rapid serial visual presentation (RSVP) protocol is of high interest because it can be used by patients who have lost control over their eyes. In this study we wish to ex- plore the suitability of recurrent neural networks (RNNs) as a machine learning method for identifying the target letter in RSVP data. We systematically compare RNN with alternative methods such as linear discriminant anal- ysis (LDA) and convolutional neural networks (CNN). Our results indicate that RNN does not have any advan- tages in single subject classfication. However, we show that a network combining CNN and RNN is superiour in transfer learning among subjects, and is significantly more resilient to temporal noise than other methods. INTRODUCTION Neural networks have recently been shown to achieve outstanding performance in several machine learning do- mains such as image recognition [15] and voice recog- nition [12]. Most of these breakthroughs have been achieved with CNNs [16], but some promising results have also been demonstrated by using RNNs for tasks such as speech and handwriting recognition [11, 10], usu- ally when using the long short-term memory (LSTM) ar- chitecture [13]. CNNs are feed forward networks that implement receptive fields. RNNs, on the other hand, contain directed cycles and are thus able to “remember” the previous activation state of the network, which makes them especially suitable for learning sequences. There have been some studies on using “deep neural net- works” for P300 classification [5, 19]. The results re- ported, despite some success, do not show the same dra- matic progress achieved by ‘deep learning’ methods as compared to the previous state of the art; while in ar- eas such as image or voice recognition ‘deep’ neural net- works have resulted in classification accuracy exceeding other methods by far, this has not yet been the case with EEG in general and P300 detection specifically. The small number of samples typically available in neuro- science (or BCI) is most likely one of the main reasons. In addition, the high dimensionality of the EEG signal, the low signal to noise (SNR) and the existence of out- liers in the data, pose other difficulties when trying to use neural networks for BCI tasks (see [18]). The main ques- tion in this research is whether the RNN model, and par- ticularly LSTM, can enhance the accuracy of P300-based BCI systems and if so, under what conditions. BACKGROUND P300-based BCI systems can recognize a taregt stimu- lus out of a set of stimuli, typically letters and numbers, by examining the subject’s EEG data. The first system that used the P300 effect was presented by [8] and since then different versions of P300 based BCI systems were suggested. One example of such a paradigm is the P300 rapid serial visual presentation (RSVP) speller. In this paradigm letters are presented one after the other in a random order, and the subject is asked to pay attention only to one of the letters, reffered to as the target (e.g., by counting them silently). There are a lot of methods for identifying the target letter for a BCI task. Blankertz et al. [4] suggest to select the time interval with maximal separation between the tar- get and non target samples, average their electro-potential value and use shrinkage LDA to classify these features. Using this method has a drawback due to the low com- plexity of LDA model [6]. The winner of the BCI com- petition III: dataset II used an ensemble of support vector machines (SVM) [21], and other methods include hidden Markov model, k-nearest neighbours, and more [6]. More recently, given the success of ‘deep’ neural net- works [15], there have been several attempts to ap- ply ‘deep learning’ for BCI related tasks. Cecotti and Graser [5] were the first to use CNNs for a P300 speller. In their work, they train an ensemble of CNN-based P300 classifiers to identify the existence of P300. Manor and Geva [19] used CNN for the RSVP P300 classification task and suggested a new spatio-temporal regularization method, which have shown improvement in the perfor- mance. Unlike feed forward network models such as CNN and multi-layer perceptron (MLP), the RNN architecture al- lows directed cycles within the network, which enable Proceedings of the 7th Graz Brain-Computer Interface Conference 2017 DOI: 10.3217/978-3-85125-533-1-87