Correcting Colours for Aided Recomposition of Fragments G. Carlomagno, F. Renna, N. Ancona, N. Mosca, G. Attolico and A. Distante Istituto di Studi sui Sistemi Intelligenti per l’Automazione, C. N. R. Via Amendola, 122 D-I –70126 - Bari Italy [renna, attolico]@ba.issia.cnr.it Abstract The paper describes the colour correction approach of a system for the virtual aided recomposition of fragmented frescos, being developed and proved for the reconstruction of the S. Matthew’s fresco, painted at the end of the 13 th century by Cimabue for the Upper Church of S. Francis in Assisi and broken in more than 140.000 fragments during the earthquake of 1997. One of its key features is the selection of fragments by a query-by-example approach applied to sample images (fragments and/or details extracted from the whole image of the fresco). Unfortunately the only available picture of the whole fresco has been acquired without any colour reference and is chromatically different across its extension and with respect to fragments: its comparison with them requires specific colour transformations to be applied to each of its regions. Two different methods to identify colour correspondences between fresco and fragments (manually and automatically from the already placed fragments) and two approaches to solve the resulting least-squares problem and evaluate the transformation matrices have been used and compared. The solution used in the system provides effective results at a very low computational cost. The restorers do not need to be aware of technical details related to the colour correction problem. The obtained improvement in terms of colour similarity is shown. 1. Introduction This work describes a component of a system for the virtual aided recomposition of fragmented frescos, whose interest arises from the need to recompose the S. Matthew fresco, painted at the end of the 13 th century by Cimabue for the Upper Church of S. Francis in Assisi and broken into more than 140.000 fragments by the earthquake in September 1997. The large extension of the fresco (about 35 squared meters), the huge number of fragments and the technique used by Cimabue (that makes the pictorial film very sensitive to the physical manipulation required by the traditional recomposition) have suggested the application of digital tools to this challenging problem. Moreover, fragments do not cover the whole fresco, partially belong to a neighbour fresco broken during the same event and exhibit contours that do not always match exactly. The system transposes the traditional recomposition process in a digital way. The operators have the critical role of applying new tools and flexible algorithms of image analysis to increase the efficacy of their work [1]. On the multi-monitor graphical station (Fig. 1) a part of the image of the whole fresco (shown in a scaled version on the central monitor) can be selected as background for the working area displayed at full resolution on the left-side monitor. The operator simultaneously rotates and translates each fragment across the region of interest using a special mouse to find its place. Virtual containers (right monitor) are the digital counterpart of the boxes used in the real lab to organize logically related fragments. Figure 1. The developed workstation for the virtual aided recomposition of fragmented frescos To improve the recomposition process, the system supports the retrieval of digital images of fragments from the database using an incremental and iterative query-by- example modality. The operator picks up a set of images (fragments and/or details of the reference image) and the system selects from the database the fragments more similar to them. This process can be repeated with different sets (by adding, removing or changing the examples) until the operator’s needs are fulfilled. Colour and texture are the most important features for similarity evaluation [3]: in fact shapes, damaged during the fragmentation process, do not necessarily match perfectly. Unfortunately, the colours of the reference image,