Enhancing Test Driven Development with Model Based Testing and Performance Analysis Sebastian Wieczorek, Alin Stefanescu SAP Research CEC Darmstadt Bleichstr. 8, 64283 Darmstadt, Germany sebastian.wieczorek@sap.com alin.stefanescu@sap.com Mathias Fritzsche SAP Research CEC Belfast Shore Road BT370QB, Newtownabbey United Kingdom mathias.fritzsche@sap.com Joachim Schnitter SAP AG Dietmar-Hopp-Allee 16, 69190 Walldorf, Germany j.schnitter@sap.com Abstract This paper suggests a holistic design and development method combining test-driven and model- driven development for SOA architectures. It uses test- driven development on component level and model- based testing on system level. Moreover, monitored performance parameters during test execution serve as input for a model-driven performance analysis of the business application, providing early indication of possible performance issues. 1. Introduction Enterprise Resource Planning (ERP) software [15] supports business processes for whole companies. Such software systems are typically very large. SAP R/3 software for instance consists of more than 250 million lines of code [16]. ERP software integrates many organizational parts and functions into one logical software system, posing unique challenges to software development and testing [22]. Enterprise Service Oriented Architecture (Enterprise SOA) [23],[18] is regarded as the next evolutionary step to cope with the software complexity of ERP systems where monolithic approaches are not applicable anymore. In Enterprise SOA, an application platform consists of independent business components exhibiting enterprise services that can be composed individually to form customized business processes. SAP recently presented a new on-demand solution named SAP Business ByDesign 1 which is fully implemented on the principles of Enterprise SOA. Since the quality of the delivered software platform is crucial for a successful application to volume business, 1 http://www.sap.com/solutions/sme/businessbydesign intensive quality assurance methods comprising several types of testing like unit, component, integration and performance are applied at SAP during the software development lifecycle. As observed by [5], the increased business process agility promoted by SOA demands a test-driven approach to develop and adhere to governance standards and hence to provide high-quality services. SAP therefore provides tools for SOA governance in its technological platform to support reusability and business alignment of SOA services developed by customers. According to SOA principles, complex application requirements are fulfilled by combining services into composite applications. This integration takes place on a higher level of abstraction than system development requiring specialized testing techniques. Test criteria like code coverage are not sufficient for service integration testing, as they do not take global communication into account but focus on local correctness. Therefore they are enhanced with more suitable criteria at higher levels of abstraction like state or transition coverage for the communication protocol between service providers and consumers [2],[13]. Model-based testing (MBT) [20] is a form of black-box testing that uses structural and behavioral models, described for instance in UML diagrams, to automatically generate test cases, thus automating the test case design process. MBT is gaining its momentum together with the model driven software development (MDD) methodology [2]. MBT test case generators aim to cover model related features, e.g. all states in an UML state machine, or data features, e.g. all boundary values. Therefore they are highly suitable for integration testing of SOA components [21]. Apart from functional correctness, software complexity also requires early identification of possible performance issues in order to avoid