Compuiers Ecfuc. Vol. 16, No. 1, pp. S-55, 1991 Printed in Great Britain. All rights reserved 0360-1315/93 $3.00 + 0.00 Copyright 0 1991 Pergamon Press plc EVALUATING TECHNOLOGY BASED TRAINING (TBT): THE ISSUES AND METHODS A. MCKEOWN Portadown College of Further Education, Portadown, Northern Ireland J. GARDNER School of Education, The Queen’s University of Belfast, Belfast BT? lNN, Northern Ireland zyxwvutsrqponmlk (Received 9 August 1990; accepted IO August 1990) Abstract-Questions are being asked about the effectiveness of technology based training in comparison to conventional training and especially about its ability to improve a company’s overall efficiency through its trainees’ performance. The need for evaluative data concerning these matters is growing and trainers themselves are revising their role as evaluators. This paper reports on a short field study which examined some of these issues. INTRODUCTION In the United Kingdom technology based training (TBT) is used in a variety of industrial and commercial settings. Barker and Yeates[I] and the Manpower Services Commission[2,3] have found that TBT has been making in roads into the training provision of the military services and the manufacturing, construction (design) and service industries. There is a wide range of training objectives, mainly for the development of basic and procedural skills but also for the less clearly defined skills of management [4]. The eady growth of TBT is illustrated by two Manpower Services Commission (now known as the Training Agency) reports. In the first, the MSC[2] found that only six of the companies approached in their survey were using TBT (all six being computer companies) and a further 11 companies were considering it. Only a year later the second survey[3] showed that a further 18 companies had taken up the techniques. In the main, however, the U.K. experience is that it is only the larger groups who are able to most fully endorse TBT (e.g.[5,6]). Reasons for this imbalance may lie in the real costs of TBT materials. In those reports which do support the cost effectiveness of TBT approaches it is often the case that insufficient account has been taken of costs such as those involved in the design and development of the materials, the provision of delivery systems and the release and supervision of the trainees. It is these types of costs which are so often underestimated both in the development of courseware and in evaluative studies of TBT usage. They are also the most onerous costs for small and medium enterprises where per capita costs tend to be substantially more than for the larger enterprises with large employee populations. For this latter group delivery costs are also more easily defrayed by ‘piggy-backing’ their courseware on existing hardware during its off-peak periods. The use of off-the-shelf courseware is not proving to be the simple solution to the small enterprises’ problems as it is often too generic to be projected easily by the trainees into their working contexts. A problem which is common to both large and small companies, whether or not they can afford the costs of generic or even bespoke courseware, is that they generally do not have any rigorous evidence of the benefits a TBT approach to their training needs may have. This problem is leading to growing demands from all sectors of industry and commerce for evaluation to provide this evidence. THE SURVEY CONTEXT At the beginning of this study, in October 1988, a literature survey [7] corroborated the view that there was only a handful of companies involved in evaluative studies of their TBT usage. For this study, seven companies were identified as being TBT users and being involved to varying degrees 51