Software process and product improvement: an empirical assessment J.P. Kuilboer, N. Ashrafi * University of Massachusetts, Management Science/Information Systems, 100 Morrissey Blvd., Boston, MA 02125, USA Received 16 March 1999; received in revised form 7 May 1999; accepted 3 June 1999 Abstract Despite all the attention that software process improvement (SPI) practices have received, there is no solid evidence of how extensively they are used across organizations, and their impact on quality, cost, and on-time delivery. The findings of previous studies are based on case studies, often assessing the effectiveness of a particular methodology in a large company. In our attempt to obtain a broader insight into the software process improvement practices, we conducted a survey targeted at software developers in New England. We collected 67 responses and used descriptive statistics to analyze the survey results. In addition, we examined the impact of SPI methodologies on quality factors and compared the impact to the importance of quality factors for software developers. The Spearman correlation coefficient was used to determine the degree of correlation between the two.  2000 Elsevier Science B.V. All rights reserved. Keywords: Software process improvement; CMM; ISO-9000; Quality factors; Empirical assessment 1. Introduction The software development crisis has been a topic of discussion for over a decade. People in the software industry have been looking for a silver bullet to solve the problems of project cancellation, cost overruns, and schedule delays. At the threshold of the millennium, as software applications grow in size, complexity, and criticality, a search for a solu- tion has become even more imperative. Over the years, a set of tools and techniques such as CASE tools, rapid applica- tion development (RAD), information engineering and many more have been undertaken. And yet, new products continue to fail to meet their functional, technical, and relia- bility objectives, often over budget and later [1,2]. Faced with these challenges, and giving up on quick-fix solutions, the software development community has been looking for a comprehensive system that provides a road- map to achieving improvements. Software process improve- ment practices (SPI) have, unquestionably, been the flavor of the 1990s [3]. SPI practices are oriented toward total process improvement rather than the final product. The assumption is that well-defined and clearly documented processes will eventually result in quality products. The International Organization for Standardization’s ISO 9000-3 standard for quality management systems, the Soft- ware Engineering Institute’s Capability Maturity Model (CMM), and the Malcolm Baldrige National Quality Award (MBNQA) are three process-oriented methodologies that provide guidelines for software process improvement. These practices do not tell software developers how to analyze, design, implement, test, or document software development. Instead, they furnish a set of standards, which provide a basis for evaluating the process of software development and eventually contribute to the continuous process improvement. The concept of SPI as an improve- ment driver is best described in an ISO report in 1992. This report states that “product quality is highly dependent on processes used in its creation, and the way toward product quality is to have available and to utilize a proven, consis- tent, reliable method of software process assessment and use the results in a continuous improvement program” [4]. Originally, NASA used SPI methodologies for develop- ing advanced military avionics applications. Today, Micro- soft, Raytheon, General Electric, and IBM are but a few companies that advertise the successful use of SPI to develop software for the external market or internal use. Mark Paulk, head of the Software Engineering Institute at Carnegie Mellon, reports that “The increasing ability of mature software organizations to deliver high-quality soft- ware products on budget and schedule shows [that the soft- ware crisis is dead]…: at least for that part of the software community that has adopted a systematic approach to soft- ware process improvement”. Despite the publicity and success stories of CMM and ISO 9000, they have not been without critics. Smaller Information and Software Technology 42 (2000) 27–34 0950-5849/00/$ - see front matter  2000 Elsevier Science B.V. All rights reserved. PII: S0950-5849(99)00054-3 www.elsevier.nl/locate/infsof * Corresponding author. Tel.: + 1-617-287-7880; fax: + 1-617-287- 7877.