Published in proceedings of 4 th Conference on Evaluation of Information Technology Delft 30-31 October 1997 Software Product Evaluation - Current status and future needs for customers and industry - Teade Punter, Rini van Solingen, Jos Trienekens Keywords: software product evaluation, software product quality, user needs, evaluation of information technology, ISO 9126, ISO 14598 ABSTRACT Evaluation of Information Technology is mostly considered as decision making on Information Technology investment proposals. However a selected investment proposal still has to be implemented. Actual realisation of such proposals is not as straightforward as it seems. Therefore activities must be started to manage conformance to the initial proposal. One of such activities is ‘Software Product Evaluation’. Software Product Evaluation is defined as the assessment of software product characteristics according to specified procedures. During a Software Product Evaluation the fit between the software product and the users’ needs of that product are determined. This fit concerns both explicit and implicit needs about the product. This is often referred to as ‘software product quality’. Software Product Evaluation becomes more popular both in industry and academics. Several evaluation techniques are already available, and others are being developed. This paper presents the current status of software product evaluation and points out the future needs. We conclude that three main areas can be distinguished on which more progress is necessary: building a quality profile, selecting appropriate actions during software implementation to address the quality profile, and designing and executing the evaluation activities. CONTENTS 1. INTRODUCTION ................................................................................................................................................................................................. 1 2. CURRENT STATUS ............................................................................................................................................................................................. 3 2.1 EXPRESSING SOFTWARE QUALITY........................................................................................................................................................................... 3 2.2 EVALUATION PROCESS ........................................................................................................................................................................................... 4 2.3 EVALUATION TECHNIQUES ..................................................................................................................................................................................... 4 2.4 SPECIFYING QUALITY REQUIREMENTS - QUALITY PROFILE ..................................................................................................................................... 5 3. FUTURE NEEDS, QUESTIONS AND PROBLEMS FOR RESEARCH......................................................................................................... 6 3.1 BUILDING QUALITY PROFILE................................................................................................................................................................................... 7 3.2 SELECTING APPROPRIATE ACTIONS TO IMPROVE SOFTWARE DEVELOPMENT .......................................................................................................... 8 3.3 DESIGNING AND EXECUTING EVALUATION ACTIVITIES ........................................................................................................................................... 9 4. CONCLUSIONS .................................................................................................................................................................................................. 10 5. ACKNOWLEDGEMENTS ................................................................................................................................................................................ 10 6. REFERENCES .................................................................................................................................................................................................... 10 About the authors: •= Teade Punter is a researcher at Eindhoven University of Technology. He is working on a PhD assignment for situated evaluation of software products. For KEMA Nederland he is involved in the SPACE-UFO project. •= Rini van Solingen is a Quality Engineer at Schlumberger Retail Petroleum Systems involved in quality measurement and improvement of embedded products. In parallel he is researcher at Eindhoven University working on a PhD Assignment for reliability improvement of embedded products and development processes. •= Jos Trienekens is an associate Professor ‘Development of IT products’ at Eindhoven University of Technology. He is responsible for a number of practice-oriented IT-research projects at the Frits Philips Institute for Quality Management (FPIQM) and Eindhoven University of Technology. He is as a consultant to KEMA Nederland involved in the SPACE-UFO project. Contact: Teade Punter Eindhoven University of Technology, Faculty of Technology Management, section Information and Technology PAV D-11, P.O. Box 513, 5600 MB Eindhoven, The Netherlands E-mail: tpu@tm.tue.nl