Towards Empirical Validation of Design Notations for Web Applications: An Experimental Framework Paolo Tonella 1 , Filippo Ricca 1 , Massimiliano Di Penta 2 , Marco Torchiano 3 1 ITC-irst, Trento, Italy 2 University of Sannio, Benevento, Italy 3 Politecnico di Torino, Italy tonella@itc.it ,ricca@itc.it, dipenta@unisannio.it, marco.torchiano@polito.it ABSTRACT Web application design involves at least one additional di- mension over traditional software design: navigation, as sup- ported by hyperlinks. Available design notations for Web applications offer enhanced separation of different design concerns (among which, navigation) and promise increased understandability and maintainability. However, such claims have not yet been tested in the field. In this paper, we propose a framework for the execution of empirical studies aimed at assessing the cost-effectiveness of Web design notations. The context of the empirical studies is a typical maintenance and evolution scenario, involving activities such as program comprehension, impact analysis and change implementation. The most important obstacles and challenges in the design of such studies will be consid- ered in this paper. We will propose counter-measures and possible mitigations for them. Finally, we will instantiate the framework into a specific empirical study that we plan to conduct in the next few months. Categories and Subject Descriptors D.2.2 [Software Engineering]: Design Tools and Tech- niques; D.2.7 [Distribution, Maintenance, and Enhance- ment]: Keywords Empirical Studies, Web Applications, Design Notations. 1. INTRODUCTION Web application design is a complex activity which requires the ability to deal with multiple and different kinds of con- cerns. A Web application is typically composed of various parts that need to be modeled at design time. Among them, the most important ones are persistent data, business logic, navigation structure, user interface. Other relevant concerns include security, transaction management, authentication. All these dimensions of a Web application must be addressed properly in the design documents. Several design notations and methodologies have been pro- posed in the literature, in an attempt to provide solutions to the problems mentioned above. Among the most refer- enced approaches are WebML [2], UWE [6], WSDM [10], OOHDM [9], Conallen [3]. Many of these notations are ex- tensions of UML [8]. Their most distinctive feature is typi- cally the ability to model explicitly the navigation structure of a Web application through a dedicated model. Such a model is often accompanied by “more traditional” entity- relationship (or similar) models (for the data), static and behavioral models (e.g., class, interaction and activity dia- grams) for the business logic, etc. Separation of concerns during Web application design is clearly important during the initial development. However, it poses many problems during the maintenance and evolu- tion phase, which actually accounts for the vast majority of an application’s life cycle[4]. In fact, it is hard to keep the different views up to date and aligned. Traceability towards the implementation may be also problematic. The overlaps and interferences between different models may be hard to detect. Overall, it might be not so obvious that the benefits encountered during the initial development are kept during the evolution phase, if assessed against the associated costs (updates, alignment, traceability, etc.). In such a context, it is extremely important to precisely un- derstand the relative merit of the various models that have been proposed in the literature, once considered during the maintenance and evolution of an existing Web application. It might be the case that some design notations are useful mainly during the initial development, while becoming only marginally useful later, with a negative cost-benefit trade off. Others might on the contrary reveal themselves as pow- erful tools that can be used to tackle the typical maintenance and evolution scenarios. Gathering such knowledge is fun- damental for the final user, who would be able to make an informed decision. However, no empirical study was con- ducted so far in this direction. In the literature, Web design methodologies are evaluated only on small examples con- structed ad-hoc by the proponents or through isolated case studies, whose results cannot be usually generalized and do not provide any comparative information.