EVALUATION OF ONTOLOGY BUILDING METHODOLOGIES A Method based on Balanced Scorecards Michela Chimienti, Michele Dassisti Dipartimento di Ingegneria Meccanica e Gestionale, Politecnico di Bari, Viale Japigia 182, Bari, Italy Antonio De Nicola, Michele Missikoff IASI-CNR, Viale Manzoni 30, Rome, Italy Keywords: Ontology Building Methodologies, Evaluation of Ontology Building Methodologies. Abstract: Ontology building methodologies concern techniques and methods related to ontology creation that starts from capturing ontology users’ requirements and concludes by releasing the final ontology. Despite the several ontology building methodologies (OBMs) developed, endowed with different characteristics, there is not yet a method to evaluate them. This paper describes an evaluation method of OBMs based on Balanced Scorecards (BSCs), a novel approach for strategic management of enterprises that we apply to the assessment of OBMs. Then, as a case study, the proposed evaluation method is applied to the UPON OBM. Finally, we show the major strengths of the BSCs’ multi -disciplinary approach. 1 INTRODUCTION An ontology is an explicit specification of a conceptualization (Gruber, 1993). Development of ontologies requires collaboration between a team of knowledge engineers (KEs) with a technical background, and domain experts (DEs) with adequate know-how in the domain to be modelled. An ontology building methodology (OBM) is a set of techniques and methods, aimed at ontology creation, that starts from capturing ontology users’ requirements and concludes by releasing the final ontology (Chimienti, 2006). Five approaches to ontology building are available (Holsapple, 2002). Using the inspirational approach, an ontology is built starting from its motivation. With the inductive approach, an ontology is built starting from observing, examining, and analyzing one or more specific cases in the domain of interest. With the deductive approach, an ontology is built starting from general principles and assumptions that are adapted and refined. Using the synthetic approach, an ontology is built starting from a base set of ontologies that are merged and synthetized. Finally, according to the collaboration approach, an ontology is built reflecting experiences and viewpoints of persons who cooperate and interact with each other. Existing OBMs usually adopt approaches that can be considered as hybrids of the five above mentioned. From a literature survey, among the most important OBMs, we cite: SENSUS methodology (Swartout, 1997), On-To-Knowledge (Sure, 2002), Ontology Development 101 (Noy, 2001), Methontology (Corcho, 2003), DILIGENT (Tempich, 2006), and UPON (De Nicola, 2009). Despite a growing literature on metrics aimed at assessing quality of ontologies (Burton-Jones, 2005), (Guarino, 2002), works related to evaluation of OBMs are still preliminary. In (Fernández-López, 1999), an approach to analyse OBMs inspired by the “IEEE 1074-1995: Standard for Developing Software Life Cycle Processes” (IEEE, 1996) is proposed. Since ontologies are part of software products, the author asserts the quality of an OBM is connected to the compliance with the processes for software development. The analysis criteria are established without defining how these should be measured and no additional perspectives, e.g., training facilities, development time, and involved human resources, are considered. (Paslaru, 2006) proposes a framework to estimate costs of ontology engineering projects, consisting of a methodology to generate a cost model, an inventory of cost drivers, and the 141 Chimienti M., Dassisti M., De Nicola A. and Missikoff M. (2009). EVALUATION OF ONTOLOGY BUILDING METHODOLOGIES - A Method based on Balanced Scorecards. In Proceedings of the International Conference on Knowledge Engineering and Ontology Development, pages 141-146 Copyright c  SciTePress