Using Concept Recognition to Annotate a Video Collection Anupama Mallik and Santanu Chaudhury Electrical Engineering Department, IIT Delhi ansimal@gmail.com, schaudhury@gmail.com Abstract. In this paper, we propose a scheme based on an ontological frame- work, to recognize concepts in multimedia data, in order to provide effective content-based access to a closed, domain-specific multimedia collection. The on- tology for the domain is constructed from high-level knowledge of the domain ly- ing with the domain experts, and further fine-tuned and refined by learning from multimedia data annotated by them. MOWL, a multimedia extension to OWL, is used to encode the concept to media-feature associations in the ontology as well as the uncertainties linked with observation of the perceptual multimedia data. Media feature classifiers help recognize low-level concepts in the videos, but the novelty of our work lies in discovery of high-level concepts in video content using the power of ontological relations between the concepts. This framework is used to provide rich, conceptual annotations to the video database, which can further be used to create hyperlinks in the video collection, to provide an effective video browsing interface to the user. 1 Introduction Meaningful access to the ever-increasing multimedia data in the pubic domain faces the crunch of available conceptual metadata and annotation text. This textual meta- data is helpful in bridging the semantic gap between high-level semantic concepts and the low-level content-based media features. Video annotation is essential for successful content-based video search and retrieval, but done manually it is tedious and prone to inaccuracy. In [1], Zha et al propose to refine video annotation by leveraging the pair- wise concurrent relation among video concepts. In [2], the authors have systematically studied the problem of event recognition in unconstrained news video sequences, by adopting the discriminative kernel-based method. Concept Recognition using an ontol- ogy for the purpose of enhancing content-based multimedia access as attempted in our work, is a relatively new approach. In our work, we propose a scheme based on an ontological framework, to recognize concepts in multimedia data, in order to generate rich, conceptual annotations for the data. The annotations generated by this scheme provide associations between the con- cepts in the domain and the content in the multimedia files, forming a basis for effective content-based access to the multimedia data in a closed, domain-specific collection. The highly specialized knowledge that experts of a scholarly domain have, is encoded into an ontological representation of the domain, and is refined by learning from observables in the multimedia examples of the domain. This approach to concept learning has been detailed in our earlier work [3]. S. Chaudhury et al. (Eds.): PReMI 2009, LNCS 5909, pp. 507–512, 2009. c  Springer-Verlag Berlin Heidelberg 2009