Visualization of topographical internal representation of learning robots Shiori Kuramoto Dept. of Applied Physics Waseda University Tokyo, Japan kuramoto@sawada.phys. waseda.ac.jp Hideyuki Sawada Dept. of Applied Physics Waseda University Tokyo, Japan sawada@waseda.jp Pitoyo Hartono School of Engineering Chukyo University Nagoya, Japan hartono@sist.chukyo-u.ac.jp Abstract— The objective of this study is to understand the learned-strategy of neural network-controlled robots in relation to their physical learning environments by visualizing the internal layer of the neural network. During the past few years, neural network-controlled robots that are able to learn in physical environments are becoming more common. While they can autonomously acquire strategy without human supervisions, it is becoming difficult to understand their strategy, especially when the robots, their environments and their tasks are complicated. In the critical fields that involve human safety, as in self-driving vehicles or medical robots, it is important for human to understand the strategies of the robots. In this preliminary study, we propose a hierarchical neural network with a two-dimensional topographical internal representation for training robots in physical environments. The 2D representation can then be visualized and analyzed to allow us to intuitively understand the input-output strategy of the robots in the context of their learning environments. In this paper, we explain about the learning dynamics of the neural network and the visual analysis of some physical experiments. Keywords— self-organizing map, explainable AI, hierarchical neural networks, reinforcement learning, autonomous robots I. INTRODUCTION The primary objective is the study to build a compact neural network for training autonomous robot that can be intuitively understood by human. In recent few years, the applications of neural networks have been proliferating in various fields. While the performances of some of them have exceeded human experts, most often they have to be treated as a black-box, in that their input-output characteristics are uninterpretable to human. The unexplanability and non- accountability of the neural networks can be problematic for their applications in some fields that involve human welfare and safety, as in self-driving vehicles, medical diagnostics and surgeries. Recently many studies are being conducted in explaining the behaviors of trained neural networks in a new field of Explainable AI (XAI) [1,2]. Many attempts to explain the strategic characteristics such as the relation between sensory inputs to a robot and its reactions that are governed by neural networks that often is difficult to understand for human, many study for understanding the input-output causality of neural networks are based on rule extractions, as in [3,4,5]. While they are successful in some cases, when the structures of the neural networks or the tasks to be learned are complicated, the rule extraction methods often generate complicated rules that are still uninterpretable for human. Hence, they do not help in increasing the accountability and transparency of the neural networks. Other attempts are to visualize some aspect of the neural networks. In this approach, the activations of parts of the neural networks, for example some of the hidden layers in hierarchical neural networks, are treated as high dimensional vectors that to some extent explain the input-output relation of the neural networks. By applying some dimensionality reduction methods [6,7,8], they can be visualized. While the visualizations do not directly generate logical rules or mathematical functions, they allow human to have intuitive understanding on the input-output relation of the neural network. The visual interpretability often generates better understandability than complicated logical rules. However, in those past studies, the methods of dimensionality reduction are often detached from the learning algorithms of the neural networks to be interpreted. The detachment of dimensionality reduction methods from the learning algorithms reduces the relation between the information to be visualized and the actual context to be explained. In this study, we experimented on learning by building a hierarchical neural network in a PC to interactively communicate with an autonomous robot in physical environment. This hierarchical neural network has a topographical hidden layer. The topographical hidden layer is two-dimensional, so that it can be visualized and intuitively analyzed, and thus allowing human to understand the strategy of the robot in relation to its learning environment. As opposed to the previous studies where the learning algorithm and the dimensionality reduction method are detached, in this study the topological dimensionality reduction is integrated into the reinforcement learning mechanism, so that the visualization reflects the actual characteristics of the neural networks. The neural network model in this study can be implemented to a small physical robot that executes a real time reinforcement learning. This study is based on the past study, in which it was reported that topological initialization, in the form of Self Organizing Maps (SOM) [9, 10], allows better learning for robots in real world environments [11]. The proposed learning method here is different from the past method, in that in the past method the topological structure is used for initializing the neural network, while in this study, the topographical self- organization is integrated with reinforcement learning mechanism, resulting in a more comprehensive visualization. The reinforcement learning in this study is based on the previous study in [12,13]. This study is different from the previous ones, in that while the reinforcement learning mechanism is identical in the previous study, the hierarchical neural network for implementing the reinforcement learning does not have low dimensional internal layer, and thus cannot be visually interpreted. In this study, we deal with collision avoidance learning of a real robot in various physical environments in real time. After the learning progressed, we visually analyzed the resulting internal low dimensional map. 978-1-7281-6926-2/20/$31.00 ©2020 IEEE