Ontology Evaluation Through Text Classification Yael Netzer, David Gabay, Meni Adler, Yoav Goldberg, and Michael Elhadad Department of Computer Science Ben Gurion University of the Negev POB 653 Be’er Sheva, 84105, Israel {yaeln|gabayd|adlerm|yoavg|elhadad}@cs.bgu.ac.il Abstract. We present a new method to evaluate a search ontology, which relies on mapping ontology instances to textual documents. On the basis of this mapping, we evaluate the adequacy of ontology relations by measuring their classification potential over the textual documents. This data-driven method provides concrete feedback to ontology main- tainers and a quantitative estimation of the functional adequacy of the ontology relations towards search experience improvement. We specifi- cally evaluate whether an ontology relation can help a semantic search engine support exploratory search. We test this ontology evaluation method on an ontology in the Movies domain, that has been acquired semi-automatically from the integration of multiple semi-structured and textual data sources (e.g., IMDb and Wikipedia). We automatically construct a domain corpus from a set of movie instances by crawling the Web for movie reviews (both profes- sional and user reviews). The 1-1 relation between textual documents (reviews) and movie instances in the ontology enables us to translate ontology relations into text classes. We verify that the text classifiers induced by key ontology relations (genre, keywords, actors) achieve high performance and exploit the properties of the learned text classifiers to provide concrete feedback on the ontology. The proposed ontology evaluation method is general and relies on the possibility to automatically align textual documents to ontology instances. 1 Introduction In this work, we present a new method to evaluate a search ontology [1]. The ontology supports a semantic search engine, which enables users to search for movies and songs recommendations in the entertainment domain. Semantic search corresponds to a shift in Information Retrieval (IR) from focus on navigational queries and document ranking to the higher level goals of content extraction, user goal recognition and content aggregation [2][3]. Our search engine operates in a limited domain (entertainment, movies). It relies on an explicit internal ontology of the domain, which captures a struc- tured representation of objects (movies, actors, directors, etc). The ontology is aquired and maintained semi-automatically from semi-structured resources (such as IMDB and Wikipedia). The ontology supports improved search experience at