A Multimedia User Preference Model that Supports Semantics and its Application to MPEG 7/21 Chrisa Tsinaraki TUC/MUSIC chrisa@ced.tuc.gr Stavros Christodoulakis TUC/MUSIC stavros@ced.tuc.gr Abstract 1 Semantic interoperability is usually provided in open environments through standards and domain on- tologies. The dominant standards for multimedia con- tent and service descriptions are MPEG-7 and MPEG- 21. The MPEG-7 Semantic DS has powerful semantic description capabilities and supports using semantic entities specified in domain ontologies in multimedia content descriptions. However, the MPEG-7/21 Usage Environment allows neither the specification of se- mantic user preferences nor the exploitation of domain knowledge and MPEG-7 semantic metadata descrip- tions. In addition, the users cannot explicitly specify, in the hierarchical MPEG-7/21 filtering and search pref- erences, the boolean operators that should be used during content filtering to combine the hierarchy com- ponents. We think these as serious limitations and we propose a hierarchical semantic user preference model that allows for the explicit specification of boolean operators. Then, we present the application of the model in MPEG-7/21 and the model implementation within the DS-MIRF framework. 1. Introduction The high penetration of the traditional TV and the advances in the digital media industry led to the devel- opment of digital multimedia content services. This paper focuses on the provision of services satisfying high-level user demands, including semantic retrieval and retrieval personalization, which are becoming in- creasingly important. Semantic interoperability is usually provided through standards and domain knowledge encoded in domain ontologies that are adopted by wide user com- munities. Domain ontologies are very important for interoperability support in open environments, as the language expressions utilizing domain ontology terms have semantic, not just syntactic, meaning for the user community. In particular, they can be used in many re- trieval aspects (like interface support, indexing, query disambiguation, etc.), thus being very useful for the 1 The work presented in this paper was partially funded in the scope of the DELOS II Network of Excellence in Digital Libraries (IST – Project Record Number 507618). semantic retrieval of multimedia data. Domain ontolo- gies are usually specified using ontology description languages like OWL [12]. Standards are necessary in open environments for interoperability support among the multimedia content services offered by different providers. The dominant metadata standards for multimedia content and service description are MPEG-7 [1] and MPEG-21 [2]. MPEG-7 provides, in the Semantic DS 2 of the MPEG-7 Multimedia Description Schemes (MDS) [4], powerful primitives for the description of complex real- world concepts and relationships among concepts. This way, the inadequacies of keyword-only descriptions (like reduced expressiveness, too many false drops etc.) are prevented. The rich structures of the MPEG-7 Se- mantic DS may also be used for both domain ontology description and multimedia content description based on domain ontologies. Such a uniform representation of ontologies and semantics within MPEG-7 allows pow- erful retrieval support. However, the utilization of ex- isting OWL domain ontologies makes interoperability support within user communities easier. A methodology for the integration of OWL domain ontologies in MPEG-7 has been developed in the DS-MIRF (Do- main-Specific Multimedia Indexing, Retrieval and Fil- tering) framework [5][6] [7][8]. A powerful retrieval API that utilizes semantic multimedia content descrip- tions has also been implemented. Although MPEG-7 allows semantic multimedia content descriptions, MPEG-7/21 allow only for lim- ited semantic user preference descriptions and they cannot currently exploit the rich semantic multimedia content descriptions formed according to the MPEG-7 Semantic DS. The user context model specified in the MPEG-21 Digital Item Adaptation (DIA) Architecture [3] and the MPEG-7 MDS for content and service per- sonalization and adaptation captures the device capa- bilities and the user-related, natural environment and network characteristics, but has two major limitations:  It cannot make use of the MPEG-7 semantic multi- media content descriptions, as it follows a keyword- based approach that may work reasonably for mov- ies, but has serious limitations in other application domains like news, sports etc. As a consequence, 2 An MPEG-7 DS (Description Scheme) essentially is an MPEG-7 complex type defined using XML Schema syntax.