Detecting GAN generated Fake Images using Co-occurrence Ma- trices Lakshmanan Nataraj; Mayachitra Inc., Santa Barbara, California, USA Tajuddin Manhar Mohammed; Mayachitra Inc., Santa Barbara, California, USA Shivkumar Chandrasekaran; Mayachitra Inc., Santa Barbara, California, USA Arjuna Flenner; Naval Air Warfare Center Weapons Division, China Lake, California, USA Jawadul H. Bappy; JD.com Amit K. Roy-Chowdhury; University of California, Riverside, California, USA B. S. Manjunath; Mayachitra Inc., Santa Barbara, California, USA Abstract The advent of Generative Adversarial Networks (GANs) has brought about completely novel ways of transforming and ma- nipulating pixels in digital images. GAN based techniques such as Image-to-Image translations, DeepFakes, and other automated methods have become increasingly popular in creating fake im- ages. In this paper, we propose a novel approach to detect GAN generated fake images using a combination of co-occurrence matrices and deep learning. We extract co-occurrence matri- ces on three color channels in the pixel domain and train a model using a deep convolutional neural network (CNN) frame- work. Experimental results on two diverse and challenging GAN datasets comprising more than 56,000 images based on unpaired image-to-image translations (cycleGAN [1]) and facial attributes/expressions (StarGAN [2]) show that our approach is promising and achieves more than 99% classification accuracy in both datasets. Further, our approach also generalizes well and achieves good results when trained on one dataset and tested on the other. Introduction Recent advances in Machine Learning and Artificial Intelli- gence have made it tremendously easy to create and synthesize digital manipulations in images and videos. In particular, Gen- erative Adversarial Networks (GANs) [3] have been one of the most promising advancements in image enhancement and manip- ulation. Due to the success of using GANs for image editing, it is now possible to use a combination of GANs and off-the-shelf image-editing tools to modify digital images to such an extent that it has become difficult to distinguish doctored images from nor- mal ones. The field of digital Image Forensics develops tools and techniques to detect manipulations in digital images such as splic- ing, resampling and copy move, but the efficacy and robustness of these tools on GAN generated images is yet to be seen. To address this, we propose a novel method to automatically identify GAN generated fake images using techniques that have been inspired from classical steganalysis. The seminal work on GANs[3] cast the machine learning field of generative modeling as a game theory optimization prob- lem. GANs contain two networks - the first network is a gener- ative network that can generate fake images and the second net- work is a discirminative network that determines if an image is real or fake. Encoded in the GAN loss function is a min-max game which creates a competition between the generative and discriminative networks. As the discriminative network becomes better at distinguishing between real and fake images, the genera- tive model becomes better at generating fake images. GANs have been applied to many image processing tasks such as image synthesis, super-resolution and image completion. Inspired by the results of these image processing tasks, GANs have brought in novel attack avenues such as computer generated (CG) faces [4], augmenting faces with CG facial attributes [2], and seamless transfer of texture between images [1], to name a few. Two of the most common applications of GANs include texture or style transfer between images and face manipulations. An example of GAN generated texture translation between im- ages, such as horses-to-zebras and summer-to-winter, is shown in Fig. 1(a). These techniques manipulates the entire image to change the visual appearance of the scene [1]. There has been also tremendous progress in facial manipulations - in particular, auto- matic generation of facial attributes and expressions. Fig. 1(b) shows one such recent example where various facial attributes such as hair, gender, age and skin color, and expressions such as anger, happiness and fear are generated on faces of celebrities [2]. While the visual results are promising, the GAN based tech- niques alter the statistics of pixels in the images that they generate. Hence, methods that look for deviations from natural image statis- tics could be effective in detecting GAN generated fake images. These methods have been well studied in the field of steganalysis which aims to detect the presence of hidden data in digital images. One such method is based on analyzing co-occurrences of pixels by computing a co-occurrence matrix. Traditionally, this method uses hand crafted features computed on the co-occurrence matrix and a machine learning classifier such as support vector machines determines if a message is hidden in the image [5, 6]. Other tech- niques involve calculating image residuals or passing the image through different filters before computing the co-occurrence ma- trix [7, 8, 9]. Inspired by steganalysis and natural image statistics, we pro- pose a novel method to identify GAN generated images using a combination of pixel co-occurrence matrices and deep learn- ing. Here we pass the co-occurrence matrices directly through a deep learning framework and allow the network to learn impor- tant features of the co-occurrence matrices. We also avoid com- putation of residuals or passing an image through various filters, but rather compute the co-occurrence matrices on the image pix- arXiv:1903.06836v2 [cs.CV] 3 Oct 2019