Abstract— We report, as part of the EMBC meeting Cognitive State Assessment (CSA) competition 2011, an empirical comparison using robust cross-validation of the performance of eleven computational approaches to real-time electroencephalography (EEG) based mental workload monitoring on Multi-Attribute Task Battery data from eight subjects. We propose a new approach, Overcomplete Spectral Regression, that combines several potentially advantageous attributes and empirically demonstrate its superior performance on these data compared to the ten other CSA methods tested. We discuss results from computational, neuroscience and experimentation points of view. I. INTRODUCTION ECENT sensor technology and analysis advances in signal processing and machine learning make it possible to noninvasively monitor brain signals and derive from them useful aspects of a person’s cognitive state in near real time [1,2]. It is now becoming feasible to integrate this technology into real-world, real-time systems to enhance human-machine interaction across a wide range of application domains including clinical, industrial, military and gaming [3,4,5,6]. However, progress in cognitive monitoring requires parallel development of new recording and analysis methods, experimental research, and empirical studies of experimental data recorded under quasi-realistic yet well- controlled operating conditions in representative subject populations. Such data allows often-neglected aspects of brain-computer interface (BCI) problems, including inter- individual differences and day-to-day variability, to be addressed. The present Cognitive State Assessment (CSA) competition 2011, its associated experimental task and accumulated data therefore provide much-needed steps toward the development of robust methods and applications. Workload measurement technology has been incrementally improved and tuned to the point where claims of near-perfect accuracy are not unheard of, despite relatively high recording noise levels, tremendous complexity of the brain, and current incomplete understanding of the underlying brain EEG signals [7,8]. There is an increasing need to compare and evaluate cognitive state estimation methods on equal footing, in particular because of the great variety of experimental tasks that have been proposed to assess different aspects of Manuscript received April 15, 2011. This work was supported in part by a gift of The Swartz Foundation (Old Field NY) and a basic research grant grant from the Office of Naval Research. C. A. Kothe and S. Makeig are with the Swartz Center for Computational Neuroscience, Institute for Neural Computation, UCSD, La Jolla CA 92093-0559 USA (corresponding author phone: +1-858-822-7538; e-mail: {christian,scott}@sccn.ucsd.edu). workload [8,9,10]. The ever-present risk of circular analysis in complex pattern recognition problems [11] also demands reliable and agreed-upon evaluation procedures for measuring estimation performance. A watershed separating the current state of the art of CSA from demonstrated robust performance in real-world settings is likely less the details of the estimation method applied, and more the amount, type and expense of the training data that is required to calibrate a predictive model capable of robust performance on later in-use data. Several factors make learning robust cognitive state estimation models difficult. First, every person has unique anatomic and functional brain geometry – both contributing to the observed inter-individual differences in the measured scalp signals. Second, because of non-reproducible sensor positioning and varying electrical conductivities at the electrode-skin interface every EEG recording session involves a sensor montage with a slightly different geometry and placement with respect to the underlying brain EEG source signals. EEG brain activity is itself highly non- stationary at all time scales (seconds to years). Thus, the further any two measurements are separated in time, the stronger the expected differences in the observed brain dynamics. To develop cognitive estimation methods that (like some recent voice recognition systems) do not require lengthy, repeated calibration or individualization, there is a need for data sets that span multiple sessions from a large number of individuals. In this initial phase of the CSA competition, the main goal is to establish a performance baseline for current state-of- the-art methods for real-time monitoring of workload. The results we present here are restricted to predictive estimation – performing a two-class discrimination task between “high” and “low” workload levels – of the performance of CSA systems that are trained on relatively short (7.5 minutes x 2 conditions), low-density (19 EEG + 2 EOG channel) data recorded from the same person on the same day and using the same montage as the data on which they are to be tested. We present preliminary performance comparisons of several state-of-the-art CSA methods including a new computational approach, introduced here, that leverages recent advances in convex optimization and statistical modeling of brain sources via a recent extension of Independent Component Analysis (ICA). Other results to be presented at EMBC 2011 will also evaluate performance of methods trained across days and montages. Estimation of Task Workload from EEG Data: New and Current Tools and Perspectives Christian A. Kothe and Scott Makeig R