Acta Informatica Malaysia 1(1) (2017) 01-06 Contents List available at RAZI Publishing Acta Informatica Malaysia(AIM) Journal Homepage: http://www.razipublishing.com/journals/acta-informatica-malaysia Teaching Performance Evaluation Framework: An Analytic Hierarchy Process Approach Azizul Azhar Ramli, Shahreen Kasim, Mohd. Farhan Md. Fuzzee, Hairulnizam Mahdin Faculty of Computer Science and Information Technology Universiti Tun Hussein Onn Malaysia Parit Raja, 86400 Batu Pahat, Johor, Malaysia {azizulr, shahreen, farhan, aizi}@uthm.edu.my ARTICLE DETAILS ABSTRACT Article history: Received 22 January 2017 Accepted 03 February 2017 Available online 05 February 2017 Keywords: Teaching Performance Evaluation Process, Multi Criteria Decision Making, Analytic Hierarchical Process Teaching performance evaluation process in higher learning institutions (HLI) is of the most importance in building a higher education policy which purposes for teaching improvement and administrative decision making. Several issues have been raised such as 1) instrument characteristic used to detect values is not sufficient to handle subjective evaluation criteria’s and, 2) the most techniques used in handling survey data is statistical method. In conventional practice, ordinal Likert scales used for marking schema and statistical techniques are implemented for analyzing the data. In order to manage the issues, this research presents a multi-criteria decision making methodology, called analytic hierarchical process (AHP) to evaluate HLI instructors teaching performance. As a well known decision making approach, AHP can systematically cope with highly subjective problems in social system. An AHP implementation framework for teaching performance evaluation will use the possible internal factors; preparation, organization, delivery and effectiveness that are evaluated as teaching evaluation weight. The future work is also suggested, this evaluation system should be implemented in real time situations. It becomes the best platform to evaluate teaching performance and categorize instructors into appropriate bands. 1. Introduction The implementation of performance evaluation is a necessary and beneficial process to provide annual feedback to staff members about job effectiveness and career guidance. The performance review is intended to be a fair and balanced assessment of an employee’s performance. Currently, there are a lot of instruments that can be used in order to assist supervisors and department heads in conducting performance reviews process in higher learning institutions (HLI) world. It is important to implement teaching performance evaluation process especially in academia. Teaching and learning are a core business and continuous improvement process must be effectively done in HLI. Otherwise those HLI will not meet their standard. In addition, HLI instructors are a back bone of this education organization. Nowadays, the evaluation criteria of teaching performance of HLI instructors usually consists of several components and parameters, each involving a number of judgments often based on imprecise data where most used instruments are an optical mark reader (OMR) answer sheets to select ordinal Likert scale answers. In addition, statistical methods are used for aggregating information from these assessment parameters. These methods are accepted especially in many HLI around the world although it is argued that this statistical technique does not necessarily offer the best way to evaluate human knowledge and skills. Hence, this study has the main objective of proposing a mechanism to evaluate the HLI instructors teaching performance evaluation. The proposed mechanism can often reflects the way people think and make judgments. With the ability and features of an MCDM approach especially AHP, in decision making process, the quality of the teaching performance evaluation will result, more reliable and validate one with highest confidences. The rest of the paper is organized as follows. In Section 2, some literature is reviewed about teaching performance evaluation process including its guideline, MCDM theory and AHP approach and also related research efforts to this research topic. Section 3 presents a suggested design model and methodology for teaching performance evaluation based on MCDM method. The final section, Section 4 presents some concluding remarks. 2. Literature Review A. Teaching Performance Evaluation Process Teaching performance evaluations are often designed to serve two purposes; to measure instructor competence and to foster professional development and growth. A teaching evaluation system should give instructors useful feedback on classroom needs, the opportunity to learn new teaching techniques and counsel from principals and other instructors on how to make changes in their classrooms. Evaluators are only mostly students. Specific procedures and standards must be first set to achieve these goals evaluations. (Boyd, 1989). Although, there are several of practices are currently applied for conducting the teaching evaluation process and with those mechanisms, there are a lot of controversies especially on the resulted reliability and validity. Furthermore, evaluation of teaching in HLI involves collecting evidence, from various stakeholders (mainly students), for the purpose of improving the effectiveness of the teaching-learning implementation process. A successful evaluation generates valid and reliable outcomes that indicate directions and actions to improve. This is a main objective of an evaluation process. Fig. 1 shows an example of an OMR form for teaching performance evaluation process with Likert scale answers. This standard evaluation form is categorized as close-ended type of questionnaires. Beside that, there are certain HLI still used open-ended questions which are including highly subjective form of evaluation criteria and need a pedagogical expertise to analyze and interpret the conclusions results. Based on stated mechanism above, OMR form notation (multiple-choice questions where the students indicate their answers by marking on the form which can then be automatically scanned and scored) are widely used because it is recognized as an efficient and cost effective data entry method for the large scale processing of category data. From its origins in the purposes of examinations, it is now widely used in other large-scale evaluations and exercises. Traditionally, questionnaires are evaluated using copies of paper questionnaires and/or OMR answer sheets. An optical mark scanner will then be used to read the marks made on the OMR forms. The process will produce an electronic data file containing responses to the questionnaires. The data file can then be analyzed using software programs such as Microsoft Excel, SPSS or SAS. With the availability of web technology, there is now the option of collecting evaluation data online. Recently, a number of IHL use Internet facilities for teaching performance evaluation purposes which is more convenient and effective to generate the final results. The conventional method recently used on teaching evaluation process especially in HLI is a statistical approach which is applied mean or average score calculation. This calculation method is synonym with the usage of an OMR form which is most often used during an evaluation process. On the other hand, there are several methods that are also applied to Cite this article as: Azizul Azhar Ramli, Shahreen Kasim, Mohd. Farhan Md. Fuzzee, Hairulnizam Mahdin, Teaching Performance Evaluation Framework: An Analytic Hierarchy Process Approach/ Acta Inform. Malay. (1) (2017) 01-06 ISSN: 2521-0874 (Print) ISSN: 2521-0505 (Online) https://doi.org/10.26480/aim.01.2017.01.06 This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.