ICSSEA 2007-11 Verner, Kitchenham & Cerpa 1/8 Estimating Project Outcomes June Verner 1 , Barbara Kitchenham 1 and Narciso Cerpa 2 1 National ICT Australia Ltd. 2 Facultad de Ingeniería, Universidad de Talca, Chile {June.Verner, Barbara.Kitchenham}@nicta.com.au, Ncerpa@utalca.cl Phone 61 2 8374 5513; Fax 61 2 8274 5520 Abstract: In spite of many years of research, many software projects still fail. As the goal of this paper is to contribute to ongoing research into the identification of factors that affect project success and failure, i.e. project outcome, we undertook a correlation study of project variables and project outcome. Our data consisted of three data sets. We developed two logistic models for the combined data selected by stepwise regression: one based on three factors and the other based on four of the raw variables. The models were compared for predictive accuracy. Twelve variables significantly associated with project success and failure, in all data sets, were identified. We found that logistic models did not predict failure well, unless the cut-off probability for classifying a project as successful corresponded to the proportion of successful projects in the data set. Only a limited number of the 12 key variables predicted project success and failure. A much larger set of variables appear not to impact project outcome. Keywords: project management, project estimation, logistic models, project success and failure 1. BACKGROUND We have been developing software since the 1960s but still have not learned enough to ensure that our IT development projects are successful. Charette suggests that “Billions of dollars are wasted each year on failed software projects” and that “We have a dismal history of projects that have gone awry” [1]. Most organizations try to hide their failures which incur not only monetary loss, but also lost opportunity. Charette provides a long list of high profile failed projects from around the world in his “Hall of Shame” [1]. Many software project success and failure factors have been described in the literature [1]-[21]. For example: organizational structure, communication with customer/users; personality conflicts; user requirements and requirements specification, scheduling and project budget, customer satisfaction; product quality; leadership; upper management support; software development methodologies; business processes and resources; and the project management process and tracking tools [1]-[21]. Some project failures are predictable and avoidable though it is not always possible to identify what success and failure factors are important early enough to take evasive action. Our research is focussed on “developing practical, actionable tools that managers can apply in the very early stages of projects” through an “understanding of why software development projects succeed or fail” [6]. This paper describes an empirical study, using data that we collected by surveying US, and Australia developers. The survey was conducted in order to investigate factors that software developers believe contribute to project success and failure (i.e. project outcome). Our contribution is to provide project managers with insight into: 1) which practices software developers believe have the greatest impact on project outcome, and hence 2) where to focus attention when resources are constrained. The research questions addressed in this paper are: 1) Which factors are most likely to impact project success and failure? 2) If we use logistic regression to predict failure is it better to use the raw variables or to reduce the variable set with factor analysis? 3) If the rate of failure and success are very different in the data sets, will changing the standard logistic regression cut-off value of 0.5 improve predictive capability.