WITCHCRAFT: EFFICIENT PGD ATTACKS WITH RANDOM STEP SIZE Tom Goldstein Renkun Ni Steven Reich Ali Shafahi Ping-Yeh Chiang Jonas Geiping Micah Goldblum University of Maryland, College Park University of Siegen ABSTRACT State-of-the-art adversarial attacks on neural networks use expensive iterative methods and numerous random restarts from different initial points. Iterative FGSM-based methods without restarts trade off performance for computational ef- ficiency because they do not adequately explore the image space and are highly sensitive to the choice of step size. We propose a variant of Projected Gradient Descent (PGD) that uses a random step size to improve performance without resorting to expensive random restarts. Our method, Wide Iterative Stochastic crafting (WITCHcraft), achieves results superior to the classical PGD attack on the CIFAR-10 and MNIST data sets but without additional computational cost. This simple modification of PGD makes crafting attacks more economical, which is important in situations like adversarial training where attacks need to be crafted in real time. Index Terms— Adversarial, Attack, PGD, CNN, CIFAR 1. INTRODUCTION Neural networks trained using stochastic gradient descent (SGD) are easily fooled by adversarial examples, small per- turbations to inputs that change the output of the network [1]. Adversarial attacks can expose serious security vulner- abilities in real-world applications such as object detection in self-driving cars [2] and classification in medical imaging [3]. In response to this threat, subsequent work has devel- oped training methods for producing neural networks robust to these attacks [4, 5]. The back-and-forth between new de- fenses and adversarial attacks that break them has spawned an array of powerful new attack methods. Among these, typical untargeted adversarial attacks oper- ate by maximizing the loss of a neural network with respect to image space, within a small ball surrounding the input, using various optimization algorithms. Targeted attacks, on the other hand, minimize loss on a particular incorrect label. In the white-box attack setting, an attacker has access to the parameters of the network, while black-box attacks operate by querying the network or transferring attacks computed on Authors contributed equally and are listed in alphabetical order. Fig. 1. WITCHcraft perturbs a “person” image into the Im- ageNet “tench” class (a kind of fish) using a targeted attack without changing the apparent class to human observers. This is an example of a “targeted” attack because the “tench” class was chosen a-priori by the attacker. The images have been blurred for anonymity. other networks. We focus on the white-box setting, a space which is dominated by optimization methods. Input spaces in computer vision are high-dimensional, and finding these small perturbations that effectively fool a network requires non-convex optimization [6]. The outputs of neural networks oscillate in these neighborhoods, so that classical gradient descent is ineffective [7], and signed gradi- ent descent methods [8] have better success. Even so, a single gradient descent is not guaranteed to solve the problem, so state-of-the-art attacks restart the attack many times with ran- dom initialization to introduce randomness and aggressively explore input space. However, this technique increases com- putational cost which may render an adversary, dynamically attacking a system in real time, useless. This is particularly problematic for adversarial training, a process in which at- tacks are generated on-the-fly during network training and used to harden a network against attacks. In this work, we develop a novel method, Wide Itera- tive Stochastic crafting (WITCHcraft), for introducing ran- domness into adversarial attacks without running the attack multiple times with different initializations. We modify the classical PGD attack, which is similar to the Basic Iterative arXiv:1911.07989v1 [cs.LG] 18 Nov 2019