An Industrial Case Study of Bypass Testing on Web Applications Jeff Offutt and Qingxiang Wang Information and Software Engineering George Mason University Fairfax, VA 22030, USA offutt@gmu.edu Joann J. Ordille Avaya Labs Research 233 Mt. Airy Road Basking Ridge, NJ 07920 USA joann@avaya.com Abstract Web applications are interactive programs that are de- ployed on the world wide web. Their execution is usually controlled very heavily by user choices and user data. This makes them vulnerable to abnormal behavior from invalid inputs as well as security attacks. Thus, web applications invest heavily in validating user inputs according to defined constraints on the values. This work focuses on validation done on the client, which uses two types of technologies; restrictions in HTML form fields and scripts that check val- ues. Unfortunately users have the ability to subvert or skip client-side validation. Bypass testing has been developed to test the behavior of web applications when client-side val- idation is skipped. This paper presents results from an in- dustry case study of bypass testing applied to a project from Avaya Research Labs, NPP. The paper presents a process for designing, implementing, automating and developing bypass tests. The theory of bypass testing had to be adapted to the unique characteristics of NPP software, which rep- resented a significant engineering challenge. The 184 tests that were generated resulted in 63 unique failures, provid- ing significant experience and numerous lessons learned. The case study also revealed several difficult problems that need to be addressed in future research. 1 Testing Web Applications Web applications are programs that are deployed on the web and that are highly user interactive [2]. Web appli- cations are composed of independent software components that can be written in heterogeneous languages and that can be distributed across multiple computers and companies. Diverse techniques and technologies are used to develop web software components and they are often integrated dy- namically [7]. They are also normally highly interactive and their executions are controlled by inputs from users. One re- sult of this is that web applications are vulnerable to security attacks that are based on manipulating inputs. The Common Vulnerability and Exposure (CVE) report of 2003-2004 [12] claimed that about 40% of security problems are a result of software faults that allow malicious code to be injected into the software via inputs. In addition, incorrect user inputs can cause software to fail. Thus, web inputs must be vali- dated to avoid data corruption, to avoid unauthorized access to sensitive information, and to prevent failures [8]. The interfaces to web applications are built with the Hy- pertext Markup Language (HTML) and can use technolo- gies such as scripting languages and browser events to re- spond to users’ actions, validate data, and prevent invalid data from being submitted. Two problems prevent the inter- face from blocking all invalid data. First, the user interface runs on the client inside a browser, allowing users to see all the validation software. Thus, users can modify HTML source and the scripts to allow unvalidated data to be sent to the server [8]. A second problem, which was not addressed by this research, is based on the stateless nature of the Hy- per Text Transfer Protocol (HTTP). Users can fool servers with multiple requests to the server by changing state in- formation that is kept in the response pages (including hid- den form fields and cookies). Both of these problems allow users to submit arbitrary data to the server. Several technologies have been used to develop web applications and validate user inputs. Some technologies run on the client, including constraints that can be defined in HTML and scripting languages that browsers interpret (such as Javascript). Server-side validation is usually im- plemented with programming languages and often use re- sources such as databases and built-in libraries. A previous paper [8] proposed the idea of bypass testing to provide invalid inputs directly to the server. This paper presents results from an industrial case study applying by- pass testing to a web application built at Avaya Research Labs, NPP. This paper is organized as follows. Section 2 presents the concepts in bypass testing. Section 3 describes the web application (NPP) that was used in the case study. Sec- 1