ISO 9126 external systems quality characteristics, sub- characteristics and domain specific criteria for evaluating e-Learning systems I. Padayachee University of KwaZulu-Natal Private Bag X54001 4000 Durban +27(031)2603525 padayacheei@ukzn.ac.za P. Kotze University of South Africa Meraka Institute, CSIR Pretoria, South Africa +27(012)8414791 paula.kotze@meraka.org.za A. van Der Merwe University of South Africa Meraka Institute, CSIR Pretoria, South Africa +27(012)8413208 alta@meraka.org.za ABSTRACT Universities and technical institutes in a developing country like South Africa are faced with the challenge of adopting and embracing a course management system (CMS) to implement e- learning. Course management systems are interactive systems that enable educators, with minimal technology expertise to design, develop and deliver e-learning content as well as measure the outcome of e-learning courses [26]. This class of software will allow these higher education institutions to stay abreast of latest educational technologies, to be competitive in the higher education domain and to afford their stakeholders new innovative ways of teaching and learning. Much of the research on CMSs has had a technology focus or has been limited to studies of adoption. Little attention has been paid to evaluating users’ views of quality of CMSs with the aid of a software quality model. This paper considers the adoption of the ISO 9126 model for selecting generic external systems quality characteristics and sub- characteristics appropriate for user evaluation of CMSs. To this end, domain specific quality criteria have been created that relate to selected characteristics and sub-characteristics of the ISO model. These quality criteria can be used to evaluate the association between end-user satisfaction (‘quality of use’ characteristic) and individual external quality sub-characteristics for CMSs. External systems quality is operationalised by 6 quality characteristics: functionality, reliability, effectiveness, usability, maintainability and portability namely and 22 sub- characteristics. The construct of ‘quality in use’ is operationalised by characteristics namely effectiveness, productivity, security and satisfaction. Categories and Subject Descriptors K.5.1 [Software Engineering}: General- standards; K.3.1 [Computers and Education]: Computer Use in Education – computer-assisted learning. General Terms Measurement, Design, Reliability, Security, Human Factors, Standardization. Keywords e-learning, external systems quality characteristics, quality in use, ISO 9126 quality model, course management systems 1. INTRODUCTION In higher education institutions, electronic teaching and learning environments have been adopted and are becoming increasingly popular among academics. These environments are popularly known as course management systems. Course management systems are interactive systems designed to enable subject matter experts, with little or no technology expertise to design, create and deliver e-learning content as well as measure the results of e- learning courses [26]. There has been limited attention given to assessing ‘quality of use ‘for course management systems despite the importance of software quality and its impact on end user satisfaction. Literature on ISO /IEC 9126 quality model was reviewed to identify quality characteristics and sub-characteristics for measuring external quality of software products. ISO/IEC 9126 categorizes quality from a user perspective as functionality, reliability, usability, efficiency, maintainability and portability. ISO/IEC 9126-2 provides 22 subcharacteristics as an operationalisation of the construct of external quality. The ISO/IEC 9126 model suggests that higher external systems quality is an indicator of higher ‘quality in use’. ‘Quality in use’ is the combined effects of the six categories of software quality when the product is used [12]. ISO/IEC 9126 defines ‘quality in use’ as ‘the capability of the software product to enable specified users to achieve specified goals with effectiveness, productivity, safety and satisfaction in a specified context of use’ [14]. ‘Quality in use’ is hence the user’s view of the quality of software operating in an environment, and is measured by the results of using the software in the environment rather than properties of the software itself. User satisfaction as a measure of software quality is often used to ascertain software success. Its common usage can be attributed to its ease of application in many areas [17]. The purpose of this paper to establish a theoretical framework comprising domain specific quality criteria relating to selected ISO /IEC 9126 systems quality characteristics and sub- characteristics pertaining to a class of software namely course management systems used in higher education institutions. This theoretical framework would be used to evaluate ‘quality in use’ of available CMSs from an educators’ perspective in a subsequent empirical study.