From Searching to Browsing through Multimodal Documents Linking Dalila Mekhaldi, Denis Lalanne and Rolf Ingold UniversitØ de Fribourg, Chemin de MusØe 3 1700 Fribourg, Switzerland {dalila.mekhaldi, denis.lalanne, rolf.ingold}@unifr.ch Abstract Relationships that link static documents discussed during meetings to the corresponding speech transcripts can be of various kinds. The most important ones, thematic links, quotations and references are presented in this paper. Thematic links are detected via a thematic alignment process. However, quotations extraction is based on the detection of segments of documents that are quoted in the speech transcript. References, made by speakers to documents, are performed via a matching process between referring expressions detected in the speech transcript, and corresponding documents logical blocks. Finally, a framework that combines these links and an evaluation of the links complementarity are presented. 1. Introduction The construction of links between related documents in collections has been widely exploited for organizing and archiving the increasing amount of documents available today [1, 2, 3]. Two levels of links exist between related documents. In the first level, called the global level, documents are considered as a whole and are linked each to other through various mechanisms (e.g. similarity, citations, hyperlinks, etc). This level is generally used for searching documents. In the second level called local level, documents are segmented into homogeneous blocks, and links are established between these blocks. While the first level has been greatly studied in information retrieval, for searching and retrieving related documents, few works focus on the second level [4]. However, the local level may be more useful and efficient than the global level for applications such as scientific conferences data archiving and indexing [5] or meetings analysis [6]. In this kind of applications, browsing interfaces are necessary for fully understanding the structure and the content of the event (e.g. conference presentation, meeting). And for this reason, local linking is required in order to align documents and play them synchronously. This alignment makes possible the linking of related parts in each independent data (static documents, slideshows and presenters audio/video recordings), transported by various media types (see figure 1). In this article, we present a case study of the local linking approach that focuses on meeting documents and the corresponding speech transcript. The linking between documents and slideshows is actually under progression. Our work is based on press reviews meetings, in which participants discuss french newspapers front pages that are composed of many heterogeneous articles. The meeting dialogs are currently transcribed manually. However, the integration of an automatic speech recognizer is under work. We will see later in this paper that, in this application, many kinds of semantic relationships ____________ _________ ____________ _________ ____________ _________ References Quotations Thematic links Speech transcript segmentation Static documents segmentation Local linking and alignment Thematic links Slides segmentation ____________ _________ ____________ _________ ____________ _________ Thematic links Speech Transcript Static documents Figure 1: Conference data local linking Slideshows