SIMULTANEOUS FORGERY IDENTIFICATION AND LOCALIZATION IN PAINTINGS USING ADVANCED CORRELATION FILTERS Paul Buchana*, Irina Cazan*, Manuel Diaz-Granados*, Felix Juefei-Xu and Marios Savvides ECE Department, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA ABSTRACT With the availability of high resolution digital technology, there has been increased interest in developing statistical and image processing techniques that can enhance the existing capabilities of analyzing works of art for authenticity. This work explores the merits of using advanced correlation filters in supplementing art experts efforts in identifying forgeries among disputed paintings. We show that by training the opti- mal trade-off synthetic discriminant function (OTSDF) filter on each section of a coarsely parceled image of an original painting, we are not only able to distinguish between a low- quality digitized representation of a painting and its forgery, but also specifically indicate where the differences occur and where the replica is particularly faithful to the original. This method is also valuable in determining whether an original painting has undergone any modifications, given that a repre- sentation of the initial version is available. Index Terms— Forgery Detection, Forgery Localization, Advanced Correlation Filters 1. INTRODUCTION Whether motivated by material gain or the desire to express admiration for another artist, art forgery has been a lucrative and active business for centuries. Traditionally, forgery detec- tion has been based on the discerning abilities of “connois- seurs”, relying on their ability to deduce authenticity from an artist’s known work, life, and influences. However, over time, these traditional methods have greatly been enhanced by quantitative methods, from X-ray analysis to isotope con- tent and most recently, mathematical tools of describing an artists style applied to high-resolution digitized versions of the paintings. A lot of progress has been made in using digi- tal techniques of feature extraction to describe an artists style. Based on these results, studies have managed to classify test paintings as originals or forgeries. However, some of these methods have shown weaknesses when faced with varying quality of the digital scans or photographs of the paintings. In this work, we describe the application of advanced cor- relation filters to (1) detecting forgeries in paintings under varying qualities of the paintings’ digital reproduction and (2) *These authors contributed equally to this work. Fig. 1: A: Originals. B: Copies. C: Image processing steps: a regular printout scan is shown. Notice on the final image a slight padding on the right side (highlighted). localizing where alterations have been made to the original. This project differs from the literature in two ways. First, it explores the merits of using advanced correlation filters in de- termining the authenticity of disputed paintings, an approach not attempted previously in this problem space. Second, it does not attempt to use an artists style signature for authenti- cation; given a known (original) work of art that was lost or stolen, we want to determine whether the version that resur- faced is indeed the original or an imitation, not to attribute never-before seen paintings to one artist or another. Previous Work: Computational image analysis and ma- chine learning techniques have been shown to be a promising way of assisting art experts in the authentication of unknown or disputed paintings. One of the most widely used tech- niques to date have involved multi-resolution analysis to extract salient features from digitized version of the paintings investigated in searching to analyze different artists’ styles. Among the feature extraction tools utilized there are wavelets ([1], [2], [3]), curvelets ([4]), craqueleure, and contourlet transforms ([5]) with the purpose of detecting subtle differ- ences in brushstrokes and painting degradation that may point to whether the painting is an original or a forgery. However, these methods are used mostly for classifica- tion, and to support art experts in the evaluation of the au- thenticity of a painting. On the other hand, correlation filters have been widely used for automatic target recognition [6], object alignment [7], biometrics recognition [8], and many other applications in which true identification is crucial, and a low margin or no false positives are allowed. To our knowl-