Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses Fu Lin (B ) , Rohit Mittapalli, Prithvijit Chattopadhyay, Daniel Bolya, and Judy Hoffman Georgia Institute of Technology, Atlanta, USA {flin68,rmittapalli3,prithvijit3,dbolya,judy}@gatech.edu Abstract. Convolutional Neural Networks have been shown to be vul- nerable to adversarial examples, which are known to locate in subspaces close to where normal data lies but are not naturally occurring and of low probability. In this work, we investigate the potential effect defense techniques have on the geometry of the likelihood landscape - likelihood of the input images under the trained model. We first propose a way to visualize the likelihood landscape leveraging an energy-based model interpretation of discriminative classifiers. Then we introduce a measure to quantify the flatness of the likelihood landscape. We observe that a subset of adversarial defense techniques results in a similar effect of flat- tening the likelihood landscape. We further explore directly regularizing towards a flat landscape for adversarial robustness. Keywords: Adversarial robustness · Understanding robustness · Deep learning 1 Introduction Although Convolutional Neural Networks (CNNs) have consistently pushed benchmarks on several computer vision tasks, ranging from image classifica- tion [15], object detection [8] to recent multimodal tasks such as visual question answering [1] and dialog [3], they are not robust to small adversarial input per- turbations. Prior work has extensively demonstrated the vulnerability of CNNs to adversarial attacks [2, 10, 24, 29, 39] and has therefore, exposed how intrin- sically unstable these systems are. Countering the susceptibility of CNNs to such attacks has motivated a number of defenses in the computer vision litera- ture [16, 18, 24, 27, 33, 35, 43, 49]. In this work we explore the questions, why are neural networks vulnerable to adversarial attacks in the first place, and how do adversarial defenses protect Electronic supplementary material The online version of this chapter (https:// doi.org/10.1007/978-3-030-66415-2 3) contains supplementary material, which is avail- able to authorized users. c  Springer Nature Switzerland AG 2020 A. Bartoli and A. Fusiello (Eds.): ECCV 2020 Workshops, LNCS 12535, pp. 39–54, 2020. https://doi.org/10.1007/978-3-030-66415-2_3