22 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA European Journal of Radiology , 1 I (1993) 22-21 EURRAD 00410 Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects C. Barillot*, D. Lemoine, L. Le Briquer, F. Lachmann, B. Gibaud INSERM U335, Luboratoire SIh4, FacultC de Midecine, UniversitC de Rennes I, 35043 Rennes Cedex, France zyxwvutsrqponmlkjihgfedcb Abstract Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combina- tion which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelhng major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data. Key worh: Images, processing; Images, analysis; Images, quality; PACS; Computers in radiology; Departmental management 1. Introduction In the last two decades, almost all processes of physics have been used to image the human anatomy. The ar- rival of new image modalities like CT Scanner, MRI, DSA, PET, or SPECT, in addition to their benefits, has resulted into a very large amount of information dif- ficult to be managed by a physician. Therefore, the classical way to deal with these data has appeared often sub-optimal with regard to their quality and to the medi- cal objective which often tend to exclude complemen- tary information from the clinical decision process. The image understanding mechanisms remain very complex and rely not only upon the matching of image data but also upon the medical knowledge about the organs and the relationships between anatomy and function. Therefore, there is actually a common statement that apart from the arrival of new image modalities, it is the arising of methods making working together the whole set of medical information that will improve the patient care. Encouraging experiments especially in 3D display have been carried out during the last 15 years to improve access and usage of medical images; however, a better *Corresponding author. use of them requires more research in data fusion. This mainly concerns the fusion of multisensor information (multimodality fusion, combination of anatomical and functional information), the fusion of information com- ing from different patients or from a priori knowledge (atlas) and finally the identification of anatomical struc- tures. Data fusion can be seen as (i) fusion of multisen- sors data (anatomical and functional information) (Fig. 1) and as (ii) fusion of multipatient data by means of warping models (Fig. 2). This paper addresses these two aspects which set the methodological framework neces- sary for the modelling of brain anatomical structures. The aim of the work presented here is to give the abili- ty to the physician to better understand the anatomical and functional environment of a lesion and to merge the relevant information to better prepare the therapeutic procedures. A direct application of this work concerns epilepsy surgery where multimodal images like CT, MRI, and DSA are used to understand the anatomical environment upon which physicians map physiology by using a priori knowledge, atlas and functional data (EEG, SEEG, MEG). Our work is an attempt to im- prove this medical process and to end up with a better accuracy by using computer procedures where physi- cians, still too often, use pencil and paper. 0720~048W93/$06.000 1993 Elsevier Scientific Publishers Ireland Ltd. All rights reserved