A Case Study Evaluation of Maintainability and Performance of Persistency Techniques Thomas Goldschmidt Software Engineering FZI Research Center for Information Technologies Karlsruhe, Germany goldschmidt@fzi.de Ralf Reussner Institute for Program Structures and Data Organisation Universität Karlsruhe (TH) Karlsruhe, Germany reussner@ipd.uka.de Jochen Winzen andrena objects ag Karlsruhe, Germany jochen.winzen@andrena.de ABSTRACT Efforts for software evolution supersede any other part of the software life cycle. Technological decisions have a major impact on the maintainability, but are not well reflected by existing code or architecture based metrics. The way the persistency of object structures with relational databases is solved affects the maintainability of the overall system. Be- sides maintainability other quality attributes of the software are of interest, in particular performance metrics. However, a systematic evaluation of the benefits and drawback of dif- ferent persistency frameworks is lacking. In this paper we systematically evaluate the maintainability and performance of different technological approaches for this mapping. The paper presents a testbed and an evaluation process with specifically designed metrics to evaluate persistency tech- niques regarding their maintainability and performance. In the second part we present and discuss the results of the case study. Categories and Subject Descriptors H.4 [Information Systems]: Miscellaneous; D.2.8 [Soft- ware Engineering]: Metrics—complexity measures, per- formance measures General Terms Measurement, Performance, Experimentation Keywords Maintainability, Performance, GQM, Persistency Techniques, Software Architecture 1. INTRODUCTION Efforts for software evolution supersede any other part of the software life cycle. Due to this fact, a large body of Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ICSE’08, May 10–18, 2008, Leipzig, Germany. Copyright 2008 ACM 978-1-60558-079-1/08/05 ...$5.00. research exists in metrics to predict and evaluate the main- tainability of software. However, often technological deci- sions also have a major impact on the maintainability, but are not well reflected by existing code or architecture based metrics. In addition, maintainability is not the only goal to optimise a software system for. Therefore, it is important to support software designers in their decision making in a way that the understand the impact of their software de- sign decisions on maintainability and other relevant quality attributes. In this paper we systematically evaluate the maintainabi- lity and performance of different technological approaches which solve the same problem, namely the mapping of ob- ject structures (as used in an object-oriented middle-tier) to a relational database (as used in the data-tier). Although this problem is certainly not new, the way the persistency of object structures with relational databases is solved affects the maintainability and the performance of the overall sys- tem. Also, this is a question which has to be solved in nearly all enterprise software applications today. Given the number of software applications used for business and administrative tasks and their life-span which often lasts over decades, the relevance of this question becomes clear relatively easy. Due to this importance of the data persistency, several approa- ches have been evolved, which are in an exemplary manner compared in this paper: 1. So called “persistency frameworks” are an established approach to persist object structures with relational database, Hibernate [4] is a popular example. The popularity of the approach is also demonstrated, by the latest version of Sun and IBM’s Enterprise Java Beans (EJB) standard where such frameworks are controlled by defined custom attributes in the Java code. As our particular testbed application is a .Net application we used the corresponding .Net implementation which is called NHibernate [5]. In addition to this open source framework we wanted to evaluate a commercial O/R mapping framework. Our choice was Vanatec Ope- nAccess [6] as representative for this category. 2. Model-driven software development (MDSD) approa- ches promise to reduce maintainability costs by raising the abstraction level to the model level. Generators can be used to create mapping definitions and classes for a persistence framework. Two different kinds of generators are possible here: 401