IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. XX, NO. XX, XX 2019 1 Generalized Learning Riemannian Space Quantization: a Case Study on Riemannian Manifold of SPD Matrices Fengzhen Tang, Mengling Fan, Peter Tiˇ no Abstract—Learning vector quantization is a simple and effi- cient classification method, enjoying great popularity. However, in many classification scenarios, such as electroencephalogram (EEG) classification, the input features are represented by sym- metric positive-definite matrices that live in a curved manifold rather than vectors that live in the flat Euclidean space. In this paper, we propose a new classification method for data points that live in curved Riemannian manifolds in the framework of learning vector quantization. The proposed method alters generalized learning vector quantization with Euclidean distance to the one operating under the appropriate Riemannian metric. We instantiate the proposed method for the Riemannian manifold of symmetric positive-definite matrices equipped with Rieman- nian natural metric. Empirical investigations on synthetic data and real-world motor imagery EEG data demonstrates that the performance of the proposed generalized learning Riemannian space quantization can significantly outperform the Euclidean generalized learning vector quantization (GLVQ), generalized relevance learning vector quantization (GRLVQ), and generalized matrix learning vector quantization (GMLVQ). The proposed method also shows competitive performance to the state-of-the- art methods on the EEG classification of motor imagery tasks. Index Terms—Learning Vector Quantization, Generalized Learning Vector Quantization, Riemannian manifold, Rieman- nian geodesic distances I. I NTRODUCTION Learning vector quantization (LVQ), introduced by Kohonen in 1986, is a prototype-based supervised classification algo- rithm based on metric comparisons of data [1]. The approach has enjoyed great popularity because of its simplicity, intuitive nature, and natural accommodation of multiclass classifica- tion problems. Unlike deep networks, the LVQ system is straightforward to interpret. The classifier constructed by LVQ is parametrized by a set of labeled prototypes living in the data space. The classification of an unknown instance takes place as an inference of the class of the closest prototype in terms of the involved metric. The learning rules of LVQ are typically based on intuitive Hebbian Learning. Thus, the implementation and realization of the method is very simple. Correspond to: Fengzhen Tang F. Tang and M. Fan are with the State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Nanta Street 114, Shenyang, 110016, China; Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110016, China; University of Chinese Academy of Sciences, Beijing 100049, China (e-mail: tangfengzhen@sia.cn,mengling@sia.ac.cn). Peter Tiˇ no is with the School of Computer Science, University of Birm- ingham, Birmingham, B15 2TT, UK(e-mail: P.Tino@cs.bham.ac.uk). Manuscript received May 20, 2019; revised December 05, 2019. Unlike many alternatives such as the perceptron or support vector machines which are in their basic form restricted to only two classes, LVQ can naturally deal with any number of classes without making the classification rule or learning algorithm more complicated. Indeed, LVQ has been used in a variety of applications such as image and signal processing, the biomedical field and medicine, and industry [2]. In Brain-Computer Interface (BCI), motor imagery is a very promising modality compared with other alternatives, as the subject voluntarily produces electroencephalogram (EEG) signal by imaging movements of different parts of the body, without external stimulus. The topological representation and band power change of brain signals during motor imagery tasks are well-known. Imaging movements of different body parts will activate (or deactivate) the activities of different area in the motor cortex of the brain, e. g. roughly, imagination of right hand movement associates with C 3 electrode, left hand C 4 , foot C z , etc [3]. Thus, in motor imagery classification, the spatial covariance matrix of the EEG signal provides enough discriminative information for different classes. This is corroborated by most commonly used Common Spatial Pattern (CSP) [4] algorithm, which is completely based on the estimation of the spatial covariance matrices, where spatial filters can be derived to enhance the class separability. Thus, the EEG signal can be represented by the corresponding sample covariance matrix, summarizing spatial information in the signal with temporal content integrated out [5]–[7]. Current LVQ and its extensions are designed to deal with data items in the form of finite dimensional real vectors. Nev- ertheless, covariance matrices are symmetric positive-definite (SPD), and as such live in a curved manifold, rather than the flat Euclidean space. The structure information of the original matrix is useful and informative for the classification task. Directly applying the existing vector based learning algo- rithms through vectorizing matrices into vectors may lead to poor generalization performance, as vectorization may destroy crucial tensor structure of matrix data. Several approaches extending vector based learning algorithms to SPD matrix data targeted for motor imagery classifcation have been sug- gested, e.g. Fisher Geodesic Discriminant Analysis (FGDA) [3], Minimum Distance to Riemannian Mean (MDRM) [8], and Tangent Space Linear Discriminat Analyisis (TSLDA) [8]. MDRM is a straightforward extension of the minimum distance to mean classification algorithm using Riemannian geometric distance and Riemannian geometric mean. It learns a cluster center (i. e. Riemannian geometric mean ) for each