A DVANCES IN THE TRAINING , PRUNING AND ENFORCEMENT OF SHAPE CONSTRAINTS OF MORPHOLOGICAL N EURAL N ETWORKS USING T ROPICAL A LGEBRA Dimitriadis Nikolaos École Polytechnique Fédérale de Lausanne Lausanne Switzerland nikolaos.dimitriadis@epfl.ch Petros Maragos National Technical University of Athens Athens, Greece maragos@cs.ntua.gr ABSTRACT In this paper we study an emerging class of neural networks based on the morphological operators of dilation and erosion. We explore these networks mathematically from a tropical geometry perspective as well as mathematical morphology. Our contributions are threefold. First, we examine the training of morphological networks via Difference-of-Convex programming methods and extend a binary morphological classifier to multiclass tasks. Second, we focus on the sparsity of dense morphological networks trained via gradient descent algorithms and compare their performance to their linear counterparts under heavy pruning, showing that the morphological networks cope far better and are characterized with superior compression capabilities. Our approach incorporates the effect of the training optimizer used and offers quantitative and qualitative explanations. Finally, we study how the architectural structure of a morphological network can affect shape constraints, focusing on monotonicity. Via Maslov Dequantization, we obtain a softened version of a known architecture and show how this approach can improve training convergence and performance. Keywords Tropical Geometry · Mathematical Morphology · Morphological Neural Networks · Monotonicity · Sparsity · Pruning · Maslov Dequantization 1 Introduction In the past decade, the field of neural networks has garnered research interest in the machine learning community, paving the way for the formation of a novel field called Deep Learning. The cell of the models is the neuron, introduced by Rosenblatt. The neuron mimics the transformations of data performed in biological organisms. In mathematical terms, it consists of a multiply-accumulate scheme that is succeeded by a nonlinearity, called activation function. An alternative lies in morphology-based models. In morphological neural networks the operations of addition and multiplication of the aforementioned multiply- accumulate scheme are replaced by maximum (or minimum) and addition, respectively. This process is called tropicalization and yields a path towards tropical mathematics. An important aspect of the operator change is the lack of need for an activation function, since maximum (or minimum) are inherently nonlinear operations. Networks with this modified neuron have been studied in the context of neural networks [1, 2, 3, 4, 5]. Recently, the field of tropical geometry has been linked with this class of morphological networks [6, 7, 8, 9, 9]. Tropical geometry studies piecewise linear (PWL) surfaces whose arithmetic is governed by a tropical semiring, where ordinary addition is replaced by the maximum or minimum and ordinary multiplication is replaced by ordinary addition. We refer to these algebraic structures as (max, +) and (min, +) semirings, respectively. These two semirings are dual and linked via the isomorphism φ(x)= −x. * This work was performed when N.Dimitriadis was at the National Technical University of Athens. arXiv:2011.07643v1 [cs.LG] 15 Nov 2020