Trainable Post-Processing Method To Reduce False Alarms In The Detection Of Small Blotches Of Archive Films Attila Licsár, László Czúni Department of Image Processing and Neurocomputing University of Veszprém Veszprém, Hungary {licsara, czuni}@almos.vein.hu Tamás Szirányi Analogical & Neural Computing Laboratory Hungarian Academy of Sciences Budapest, Hungary sziranyi@sztaki.hu Abstract—We have developed a new semi-automatic neural network based method to detect blotches with low false alarm rate on archive films. Blotches can be modeled as temporal intensity discontinuities, hence false detection results originate from object motion (e.g. occlusion), non-rigid objects or erroneous motion estimation. In practice, usually, after the automatic detection step the false alarms are removed manually by an operator, significantly decreasing the efficiency of the restoration process. Our post-processing method classifies each detected blotch by its image features to minimize false results and the necessity of human intervention. The proposed method is tested on real archive sequences. Keywords-digital film restoration; blotch detection; machine learning I. INTRODUCTION In national film archives there are huge amounts of archive films to be restored. These films suffer from several degradations such as blotches, scratches, flickering (intensity fluctuation), image vibration (displacement of adjacent frames), fading, discoloring, etc. Besides traditional analog techniques, semi-automatic digital restoration methods provide an efficient way to achieve cost efficient saving and reconstruction of the film heritage, i.e. fast, robust and automatic processing with a minimal human invention. Some types of very annoying errors are called one frame defects and they are mostly visible as blotches. These artifacts appear at random positions on consecutive frames and with high contrast against the background. They have arbitrary shape, size and varying range of intensity (from bright to dark). Blotches are usually caused by dirt, damage of the film surface and chemical or biological processes such as mold. Blotches can be modeled as temporal discontinuities of pixel intensity not originating from object motion (occlusion, disocclusion) or non-rigid objects. A typical restoration procedure of one-frame defects is the following [9]: (1) detection of the defected regions, (2) interpolation of the corrupt image regions by spatio-temporal inpainting methods. In practice, after the automatic detection an operator manually verifies and corrects false results. In case of lots of false alarms the latter step is time consuming and results in a bottleneck of the restoration process. Our paper deals with an automatic detection step and with the minimization of the human intervention. The tuning of the detection parameters gives a trade-off between high correct detection and low false alarm rate (when an object is wrongly detected as artifact). In general, we prefer lower false alarm rate rather than high detection rate because the replacement of a real object with any inpainting, due to false detections, causes loss of original image details (e.g. buttons on the clothes) not acceptable by archivists. Hence an automatic method is needed to reduce false alarms of the previously detected blotches by classifying them. Serious problems are the influence of local/global motion and the presence of other film degradations such as vibration or flickering that yield false alarms in the detection phase. Further difficulty is the huge amount of data to be processed (e.g. motion estimation) at high resolution processing (2000x1500 pixels (2K) or higher) of 35mm archive films. Our paper presents a blotch detection method with hierarchical gradient-based motion estimation with low computational cost. Optical flow calculation reduces false alarm detection rate owing to the object motion or image vibration. This step contains a preliminary detection step that speeds up the computation time of the optical flow by a pre- selection of the regions to be processed. Our main result is the improvement of detection efficiency during post-processing by a feature based neural network (NN) classification. This is essential to achieve a cost effective and efficient restoration by the reduction of the human intervention. II. OTHER WORKS Main approaches of blotch detection methods are in the following two groups: (1) detection by analysis of contrast or local maxima/minima; (2) methods based on the detection of temporal discontinuities. The first group includes morphological operator based methods [12],[7],[14],[4] resulting in low complexity because they do not require temporal analysis such as motion estimation. Methods in the second group are based on the detection of temporal discontinuities like the SDI (Spike Detection Index) [9], ROD (Rank Ordered Differences) [11][5], MRF (Markov Random Field) [8] methods. According to the comparative evaluation of 0-7803-9134-9/05/$20.00 ©2005 IEEE II-562