Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classification Anh Huy Phan a,Ã , Andrzej Cichocki a,b a Lab for Advanced Brain Signal Processing, BSI, RIKEN, Wakoshi, Saitama, Japan b Systems Research Institute Polish Academy of Science, Poland article info Available online 19 February 2011 Keywords: Tensor decompositions Tucker decomposition Tucker-3 Tucker-2 DEDICOM models Alternating least squares (ALS) Hierarchical alternating least squares (HALS) Large-scale dataset Multiway classification Social networks analysis abstract Analysis of high dimensional data in modern applications, such as neuroscience, text mining, spectral analysis, chemometrices naturally requires tensor decomposition methods. The Tucker decompositions allow us to extract hidden factors (component matrices) with different dimension in each mode, and investigate interactions among various modalities. The alternating least squares (ALS) algorithms have been confirmed effective and efficient in most of tensor decompositions, especially Tucker with orthogonality constraints. However, for nonnegative Tucker decomposition (NTD), standard ALS algorithms suffer from unstable convergence properties, demand high computational cost for large scale problems due to matrix inverse, and often return suboptimal solutions. Moreover they are quite sensitive with respect to noise, and can be relatively slow in the special case when data are nearly collinear. In this paper, we propose a new algorithm for nonnegative Tucker decomposition based on constrained minimization of a set of local cost functions and hierarchical alternating least squares (HALS). The developed NTD-HALS algorithm sequentially updates components, hence avoids matrix inverse, and is suitable for large-scale problems. The proposed algorithm is also regularized with additional constraint terms such as sparseness, orthogonality, smoothness, and especially discriminant. Extensive experiments confirm the validity and higher performance of the developed algorithm in comparison with other existing algorithms. & 2011 Elsevier B.V. All rights reserved. 1. Introduction In many applications such as data analysis in neuroscience, the data structures often contain high-order ways (modes) including trials, task conditions, subjects, together with the intrinsic dimen- sions of space, time, and frequency. Analysis on separate matrices or slices extracted from a data block often faces the risk of losing the covariance information among various modes. To discover hidden components within the data, the analysis tools should reflect the multidimensional structure of the data [5]. Tensor decompositions are emerging as novel and promising tools for exploratory analysis of multidimensional data in diverse disciplines including text mining, neuroscience, computer vision, and social networks analysis [5–8]. Tensor decompositions cap- ture multilinear structures in higher-order datasets, where data have more than two modes [9–13]. Tensor decompositions and multiway analysis allow naturally to extract hidden (latent) components and investigate complex relationships among them, for example, in exploration of social networks [8,14,15]. One of the most common tensor decomposition is the Tucker decomposition which was first introduced by Tucker [16]. The Tucker decomposition and its variation have already found many applica- tions, for example, in signal processing [17], extended Wiener filters [18], face recognition using Tensor-faces [19], analysis of multichannel EEG and MEG data [20]. A particular case of the constrained Tucker decomposition is the DEDICOM (DEcomposition into DIrectional COMponents) model [21, 22] which allows to analyze asymmetric relationships between groups (objects), for example, in social networks. In this direction, Harshman and Lundy [22] analyzed asymmetric measures of yearly trade (import–export) among a set of nations over a period of 10 years. Lundy et al. [23] presented an application of three-way DEDICOM to skew-symmetric data for paired preference ratings of treatments for chronic back pain (with additional constraints to obtain meaningful results). Bader et al. analyzed email communications of Enron company using the three- way DEDICOM model [12, 24]. The general Tucker model does not impose any constraint on factors and a core tensor. In many applications such as dimensionality reduction, feature extraction, several existing Tucker decomposition algorithms consider orthogonality of factors, such as the higher-order singular value decomposition (HOSVD), higher order orthogonal iteration (HOOI) algorithms [25–28]. For nonnegative Tucker Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/neucom Neurocomputing 0925-2312/$ - see front matter & 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.neucom.2010.06.031 Ã Corresponding author. E-mail addresses: phan@brain.riken.jp (A.H. Phan), cia@brain.riken.jp (A. Cichocki). Neurocomputing 74 (2011) 1956–1969