50 Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Chapter 4 INTRODUCTION The terms “Open Source Software” (OSS), “Free Software”, “Free Open Source Software” (FOSS), and “Free/Libre Open Source Software” (FLOSS) are often treated synonymously (Feller & Fitzger- ald, 2002; Feller et al., 2005; Koch, 2005). When we look at their respective license agreements, however, we can easily see that they are quite different. Free software is generally licensed with the GNU General Public License (GPL), while OSS may use either the GPL or some other license that allows for the integration of software that may not be free (Elliott & Scacchi, 2008; Gay, 2002). Free software is always available as OSS, but OSS is not always free software. Therefore it Norita Ahmad American University of Sharjah, UAE Phillip A. Laplante The Pennsylvania State University, Great Valley, USA A Systematic Approach to Evaluating Open Source Software ABSTRACT Selecting appropriate Open Source Software (OSS) for a given problem or a set of requirements can be very challenging. Some of the difficulties are due to the fact that there is not a generally accepted set of criteria to use in evaluation and that there are usually many OSS projects available to solve a particular problem. In this study, the authors propose a set of criteria and a methodology for assessing candidate OSS for fitness of purpose using both functional and non-functional factors. The authors then use these criteria in an improved solution to the decision problem using the well-developed Analytical Hierarchy Process. In order to validate the proposed model, it is applied at a technology management company in the United Arab Emirates, which integrates many OSS solutions into its Information Technology infra- structure. The contribution of this work is to help decision makers to better identify an appropriate OSS solution using a systematic approach without the need for intensive performance testing. DOI: 10.4018/978-1-4666-2782-6.ch004