An Automated Testing Environment for CTI Systems Using Concepts for Specification and Verification of Workflows Oliver Niese 1 , Tiziana Margaria 1 , Andreas Hagerer 1 , Markus Nagelmann 1 , Bernhard Steffen 2 , Georg Brune 3 and Hans-Dieter Ide 3 1 METAFrame Technologies GmbH, Dortmund, Germany {ONiese, TMargaria, AHagerer, MNagelmann}@METAFrame.de 2 Chair of Programming Systems, University of Dortmund, Germany Steffen@cs.uni-dortmund.de 3 Siemens AG, Witten {Georg.Brune, Hans-Dieter.Ide}@wit.siemens.de Abstract. In this paper we present an automated testing environment for the system level test of Computer Telephony Integrated applications. Its novelty consists of a coordinative test management layer that instantiates a general-purpose environment for the specification and verification of workflows in the testing domain. This results in a test environment that does not only control the in- dividual test tools but the whole life-cycle of functional system-level tests, comprising test design, test generation, test execution, test evaluation and test reporting. Special attention is devoted to the simpli- fication of the test cases design and the checking of admissibility criteria, interdependencies between the actions of test cases, and assumptions about the state of the system's resources. The key features of our testing environment, coarse grain, workflow-like test case representation and the validation and formal verification capabilities, are illustrated along a concrete industrial application. 1 Introduction The increasing complexity of today's testing scenarios for telephony systems demands for an integrated, open and flexible approach to support the management of the overall test process, i.e. specification of tests, execution of tests and analysis of test results. Furthermore, systems under test (SUT) become com- posite (e.g. including Computer Telephony Integrated (CTI) platform aspects), embedded (due to hard- ware/software codesign practices), reactive, and run on distributed architectures (e.g. client/server archi- tectures). As a consequence, it becomes increasingly unrealistic to restrict the consideration of the testing activities to single units of the systems, since complex subsystems affect each other and require scalable, integrated test methodologies. The requirements discussed in this paper exceed the capabilities of today's testing tools. To our knowl- edge there exist neither commercial nor academic tools providing comprehensive support for the whole system-level test process. In particular, most research on test automation for telecommunication systems concentrates on the generation of test cases and test suites on the basis of a formal model of the system: academic tools, like TORX [12], TGV [3], Autolink [5], as well as commercial ones like Telelogic Tau [10] presuppose the existence of fine-granular system models in terms of either automata or SDL de- scriptions, and aim at supporting the generation of corresponding test cases and test suites. This approach failed to enter practice in the scenario we are considering here, because it did not fit the current system design practice. In order to overcome this hurdle, we aim at a formal methods-controlled, component-based test design on top of a library of elementary but intuitively understandable test case fragments. This establishes a coarse- granular ‘meta-level’ on which - test engineers are used to think, - test cases and test suites can be easily composed, - test scenarios can be configured and initialized,