260 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 36, NO. 2, MARCH 2006 Evaluation of Enterprise Information Technologies: A Decision Model for High-Level Consideration of Strategic and Operational Issues Joseph Sarkis and R. P. Sundarraj Abstract—Despite the possibility of short-term financial losses and concerns by executive management of potential failure, enter- prise information technologies (EITs) are being implemented by a growing number of Fortune 100 and midsize corporations in the hope of acquiring long-term benefits. EITs, expensive and risky in- formation technology assets, must be evaluated on the basis of both their productivity gains and their support for corporate reengi- neering through integration of business processes. In this paper, we propose a two-stage methodology (involving a combination of the analytical network process and integer programming) to conduct a high-level evaluation of an interdependent set of tangible, intan- gible, strategic and operational factors that should be considered in EIT evaluation. We perform sensitivity analyses and conduct tests with real-world data to show the robustness of our methodology and the resulting managerial implications. Index Terms—Analytical hierarchical process (AHP), analyt- ical network process (ANP), enterprise information technologies (EITs), enterprise resource planning (ERP), integer programming (IP). I. INTRODUCTION A S information technologies (ITs) become increasingly expensive to purchase and maintain, organizations are facing critical questions from stakeholders, and are requiring their managers to provide better justification of their systems [67]. This concern is particularly true of enterprise information technologies (EITs). 1 EIT systems often cost millions of dollars and take years to implement. Thus, they must be evaluated from a strategic perspective. However, an EIT system also has wide-scale operational-level usage, and the lack of effective use at this level can sometimes result in organization-wide difficulties [5], [11]. Therefore, a project manager evaluating an EIT system must consider operational issues as well, albeit at a high level of abstraction. Unfortunately, research work on such a comprehensive evaluation is scarce. This paper provides a two-stage methodology for a high-level assessment of EIT systems. Our first stage is an application Manuscript received September 18, 2001; revised April 5, 2002 and Novem- ber 5, 2004. This paper was recommended by Associate Editor J. Malmberg. J. Sarkis is with the Graduate School of Management, Clark University, Worcester, MA 01610-1477 USA (e-mail: jsarkis@clarku.edu). R. P. Sundarraj is with the Department of Management Sciences, University of Waterloo, Waterloo ON N2L 3G1, Canada (e-mail: rsundarr@engmail.uwa- terloo.ca). Digital Object Identifier 10.1109/TSMCC.2004.843245 1 Although similar to an enterprise resource planning (ERP) system, an EIT as considered herein is slightly broader in scope. EITs encompass ERP as well as technologies for advanced decision support for which conventional ERP prod- ucts do not generally have a sophisticated solution. of the analytical network process (ANP) method, which pro- duces utility weights for the EIT alternatives, based on deci- sion-maker preferences. ANP is similar to the analytical hier- archical process (AHP) technique, and it is particularly suited for EIT evaluation, because of its ability to handle interdepen- dencies among factors. Moreover, ANP has seen limited use as a decision making tool. We test the ANP model using data from Texas Instruments (TI). In the second stage, the utility weights from ANP are then considered in an integer programming (IP) model that selects a set of alternative(s), subject to managerial and budget constraints. Significant managerial insights can be gained from sensitivity analyses with both of these models. The methodology supports a hierarchical relationship, linking upper management with a critical overview of operational issues. It incorporates tangible and intangible factors, taking into consid- eration strategic and operational requirements. We begin with a brief review of the literature on evaluation of IT and other relevant technologies. Section III overviews factors that are needed in an EIT evaluation and briefly describes the proposed approach to deal with these factors. Details of the ANP methodology and of its testing on TI are provided in Sections IV and V,respectively, while Section VI presents an IP model that incorporates the ANP information. We complete the paper with a summary and review of some managerial and research issues. II. BACKGROUND A. EIT Systems EIT systems are packaged software systems that include em- bedded business models of industrys “best practices” [62]. In this sense, they represent a hybrid approach between costly cus- tomized software, and inflexible packaged software [56]. An EIT typically includes a number of systems which “bolt-on” to an enterprise resource planning (ERP) system. These systems may offer advanced supply-chain planning capabilities that enable global optimization, manage transporta- tion, sales-force, warehousing and customer-related activities, conduct automated business-to-business (B2B) e-commerce transactions with customers and suppliers, and support vir- tual-enterprise activities [11]. Because of these interconnections, a single entry into an EIT system triggers a whole set of processes that help with both macro- and microlevel decision making. TI manages complex global and multiorganizational customer needs through a suite of three tightly integrated systems: SAP’s ERP system, i2’s ad- vanced-planning optimizer, and a shop floor scheduling system. 1094-6977/$20.00 © 2006 IEEE