Interpretable Neuron Structuring with Graph Spectral Regularization Alexander Tong 1 , David van Dijk 2 , Jay S. Stanley III 2 , Matthew Amodio 1 , Kristina Yim 2 , Rebecca Muhle 2 , James Noonan 2 , Guy Wolf 3 , and Smita Krishnaswamy 1,2(B ) 1 Yale Department of Computer Science, New Haven, USA smita.krishnaswamy@yale.edu 2 Yale Department of Genetics, New Haven, USA 3 Department of Mathematics and Statistics, Universit´ e de Montr´ eal, Mila, Montreal, Canada Abstract. While neural networks are powerful approximators used to classify or embed data into lower dimensional spaces, they are often regarded as black boxes with uninterpretable features. Here we pro- pose Graph Spectral Regularization for making hidden layers more inter- pretable without significantly impacting performance on the primary task. Taking inspiration from spatial organization and localization of neu- ron activations in biological networks, we use a graph Laplacian penalty to structure the activations within a layer. This penalty encourages acti- vations to be smooth either on a predetermined graph or on a feature- space graph learned from the data via co-activations of a hidden layer of the neural network. We show numerous uses for this additional struc- ture including cluster indication and visualization in biological and image data sets. Keywords: Neural Network Interpretability · Graph learning · Feature saliency 1 Introduction Common intuitions and motivating explanations for the success of deep learning approaches rely on analogies between artificial and biological neural networks, and the mechanism they use for processing information. However, one aspect that is overlooked is the spatial organization of neurons in the brain. Indeed, the hierarchical spatial organization of neurons, determined via fMRI and other technologies [13, 16], is often leveraged in neuroscience works to explore, under- stand, and interpret various neural processing mechanisms and high-level brain functions. In artificial neural networks (ANN), on the other hand, hidden layers offer no organization that can be regarded as equivalent to the biological one. This lack of organization poses great difficulties in exploring and interpreting A. Tong, D. Dijk, G. Wolf and S. Krishnaswamy—Equal contribution. c  The Author(s) 2020 M. R. Berthold et al. (Eds.): IDA 2020, LNCS 12080, pp. 509–521, 2020. https://doi.org/10.1007/978-3-030-44584-3_40