Accepted to The 10th European Conference on Information Technology Evaluation (ECITE-2003), 25-26 September 2003, Madrid Six Generic Types of Information Systems Evaluation Stefan Cronholm, Göran Goldkuhl Department of Computer and Information Science Linköping University E-mail scr@ida.liu.se , ggo@ida.liu.se Abstract The aim of this paper is to contribute to the decision of how to perform evaluation depending on the evaluation context. Three general strategies of how to perform evaluation together with two general strategies of what to evaluate are identified. From the three “how-strategies” and the two “what-strategies” we derive a matrix consisting of six generic types of evalua- tion. Each one of the six types are categorised on a ideal typical level. Keywords: IS Evaluation, IS Assessment, Information Systems, Goal-based evaluation, Goal-free evaluation, Criteria-based evaluation 1. Introduction All over the world there is a huge amount of money spent on IT (e.g. Seddon, 2001). It is therefore important to evaluate the outcome. Evaluation is never an easy task and conse- quently there are a lot of suggestions for how to evaluate IT-system. Much of the literature on evaluation takes a formal-rational view and sees evaluation as a largely quantitative process of calculating the likely cost/benefit on the basis of defined criteria (Walsham, 1993). There are also interpretative approaches (e.g. Remenyi, 1999; Walsham, 1993). The interpretative perspective views IT-systems often as social systems that have information technology em- bedded into it (Goldkuhl & Lyytinen, 1982). There are formative and summative approaches containing different measures or criteria. Some approaches are focusing on harder economical criteria and others are focusing on softer user-oriented criteria. According to Walsham (1993) and Scriven (1967) formative evaluation aims to provide systematic feedback to the designers and implementers while summative evaluation is concerned with identifying and assessing the worth of programme outcomes in the light of initially specified success criteria after the implementation of the change pro- gramme is completed. The criteria used are often derived from one specific perspective or theory. All of the approaches, formal-rational, interpretative or criteria-based, are different ways and their primary message is how the evaluator should act in order to perform evaluation. Besides this “how-message” it is also important to decide about what to evaluate. When evaluating IT-systems we can think of at least two different situations that can be evaluated. In this pa- per, we differ between evaluation of IT-system as such and evaluation of IT-systems in use. From the questions of how to evaluate and what to evaluate we derive a matrix consisting of two dimensions “how to evaluate” and “what to evaluate”. The combination of the two di- mensions results in six different evaluation types and the purpose of this paper is to, on an ideal typical level, identify and characterise each of the derived evaluation types. The aim of the matrix is to support different choices of how to perform an evaluation depending on the evaluation situation. 1