BAYESIAN FRAMEWORK FOR WHITE MATTER FIBERS SIMILARITY MEASURE D. Wassermann 1 , L. Bloy 2 , R.Verma 2 , R. Deriche 1 1 INRIA Sophia Antipolis - Mediterran´ ee, Odyss´ ee Project Team, 2004 Route des Lucioles, Sophia-Antipolis, 06902, France 2 University of Pennsylvania , Radiology, Philadelphia, PA 19104, USA ABSTRACT We provide a Bayesian framework for measuring similarity in white matter fiber bundles based on Gaussian Processes. This framework does not rely on point-to-point correspondences, it takes into account a priori information about the fiber struc- ture working with three dimensional curves instead of point sequences. Moreover, it spans an inner product space among curves together with its induced metric. Thus, it provides an environment to perform statistics on curves. Finally, we show clustering results to illustrate the utility of this model. Index Terms— Magnetic resonance imaging, Tractogra- phy, Gaussian processes, Mode seeking 1. INTRODUCTION Diffusion MRI non invasively recovers the in vivo effective diffusion of water molecules in biological tissues, thus pro- viding unique biologically and clinically relevant information not available from other imaging modalities. This informa- tion can help characterize tissue micro-structure and its ar- chitectural organization [1] by modeling the local anisotropy of the diffusion process of water molecules. Once the diffu- sion information has been recovered within each voxel, it can be assembled throughout the volume in order to assess brain connectivity in vivo, one of the prominent techniques to ac- complish this is streamline tractography [2], which recovers white matter fiber tracts from a seed voxel by following the principal direction of the diffusion tensor. Multiple seeds can be scattered in the brain resulting in a full brain reconstruc- tion of the fiber tracts. However, analysis or visualization of the whole fiber ensemble in order to get insight of the brain structure is difficult due to cluttering of the fibers as depicted in figure 3. Thus, there is a need for automatic fiber cluster- ing algorithms in order to perform visualization, anatomical structure identification and group analysis. A main issue ren- dering automatic clustering a challenge is the fact that axons This work was partly supported by the INRIA International Dept. (CORDIS Program and CD-MRI Associated team) and the EADS Founda- tion (EADS n o 2118). L. Bloy is supported by NIH grants T32-EB-000814 and RO1MH079938 and R. Verma by RO1MH079938. Data was collected as part of NIH grants R01MH079938 and R01MH060722. integrating a bundle can also diverge from it connecting cor- tical and subcortical areas [3]. Due to this, approaches that quantify similarity among fibers through usual shape statis- tics or rigid transformations are unsuited. Take for instance the cingulum, whose constituent fibers only partially overlap, with many diverging to inervate the cortex. These divergent fibers can have quite different shapes, calling into question the utility of shape based metrics. Quantifying fiber similarity is a fundamental element of fiber clustering that has been addressed in different ways and with different purposes. The works by [4, 5, 6] quantify fiber similarity with different flavors of shape statistics. However, partial overlapping of fibers is not taken into account as a similarity feature rendering the approaches unsuited for auto- matic classification of fibers in the brain. On the other hand, the works [7, 8, 9, 10], use different clustering algorithms based on the Hausdorff or Chamfer distances among the se- quence of points representing each fiber tract. This family of similarity metrics deal with sets of points instead of curves, hence they discards continuity or directionality information. Moreover, similarity tends to decrease very fast in cases of partial overlapping, clustering out fibers diverging from the bundle. In particular, [7] only analyzes fibers whose seed points are spatially close together. This is not suited for a whole brain analysis, where different fiber seed points from the same bundle might have been scattered over the brain in order to overcome streamline tracking limitations on complex fiber configurations [8]. Manifold learning techniques are used in [8, 9] to generalize this type of distances from small sets of simlilar fibers to a bigger more diverse set of fibers. This approach embeds the fibers into euclidean or topologi- cal spaces which can be handled more easily. However, this strategy has produced limited results. In [8] an atlas is gen- erated and fibers from new subjects are classified according to the atlas. Even though these automatically grouped bun- dles are anatomically coherent, the atlas generating process requires heavy user interaction and fine parameter tuning ren- dering it difficult to be reproduced. In [9], a publicly available anatomical atlas is used in conjunction with the fiber similar- ity metric and a smaller amount of parameters is needed, nev- ertheless situations of partial fiber overlapping generate non- anatomically coherent bundles. In [10] the Hausdorff similar- 815 978-1-4244-3932-4/09/$25.00 ©2009 IEEE ISBI 2009