Keep breathing! Common motion helps multi-modal mapping V. De Luca ⋆1 , H. Grabner 1 , L. Petrusca 2 , R. Salomir 2 , G. Sz´ekely 1 , and C. Tanner 1 1 Computer Vision Laboratory, ETH Z¨ urich, 8092 Z¨ urich, Switzerland 2 Radiology Department, Geneva University Hospital, Geneva, Switzerland Abstract. We propose an unconventional approach for transferring of information between multi-modal images. It exploits the temporal com- monality of multi-modal images acquired from the same organ during free-breathing. Strikingly there is no need for capturing the same region by the modalities. The method is based on extracting a low-dimensional description of the image sequences, selecting the common cause sig- nal (breathing) for both modalities and finding the most similar sub- sequences for predicting image feature location. The approach was eval- uated for 3 volunteers on sequences of 2D MRI and 2D ultrasound images of the liver acquired at different locations. Simultaneous acquisition of these images allowed for quantitative evaluation (predicted versus ground truth MRI feature locations). The best performance was achieved with signal extraction by slow feature analysis resulting in an average error of 2.6 mm (4.2 mm) for sequences acquired at the same (a different) time. Keywords: motion prediction, multi-modal, ultrasound, magnetic res- onance images, tracking, liver 1 Introduction The attention paid to organ motion due to breathing during radiation therapy has risen in recent years [4]. As treatment devices become more sophisticated, their guidance requires higher accuracy. Yet motion prediction is complicated due to the large variability in respiratory patterns, invalidating the assumption of periodic motion [4]. Unfortunately, observation of the tumor motion in real-time during therapy is often impossible, making surrogate measures of this motion essential. Common surrogates include measurements from a breathing bellow, a spirometer, optical imaging of the chest wall and imaging of the diaphragm position. The latter could be achieved by fluoroscopy, ultrasound or cine MRI, with ultrasound being the preferred choice to avoid radiation and high costs. The question remains how well the surrogates can be related to the tu- mor motion. Studies have shown that assuming a linear relationship between the abdominal and tumor displacements can be misleading (correlation range ⋆ We acknowledge the Swiss National Science Foundation - Project CRSII2 127549.