E-University International Conference on Engineering Education August 18–22, 2002, Manchester, U.K. 1 KNOWLEDGE BASED EXPERT SYSTEM FOR ON-LINE ORAL EXAMINATION Pramod Kachave 1 , Ajit More 2 , S.G.Bhirud 3 and Charudatta Pathak 4 1 Pramod Kachave, SSVPS BSD Polytechnic, Dhule 424 005 INDIA prmodkachave@rediffmail.com 2 Ajit More, SSVPS Engineering College, North Maharashtra University, Jalgaon 425 001 INDIA ajitmmore@rediffmail.com 3 S.G.Bhirud, VJTI, Mumbai INDIA sgbhirud@yahoo.com 4 Charudatta Pathak, SSVPS Engineering College, North Maharashtra University, Jalgaon 425 001 INDIA charudatta_pathak@yahoo.com Abstract  Knowledge based on-line examination system is backbone for any E-University. This paper discusses the scope and structure of an oral examination system in era of electronic assessment. The main goal while developing this structure was to remove drawbacks of existing oral examination in which lot of factors involves in marking the performance of students. Attempt has also been made to capture operating procedure of an impartial examiner, who is treated as expert, in logical form. In this system, the question bank will content questions in primary, basic and expert categories. Students appearing for the oral examination will get the questions from the server program who is domain expert. Initially server will distribute questions randomly from primary knowledge category. Answers from the students are recorded and analyze by the server. Successful students from the primary category will get questions randomly from the basic and expert category successively. Students will get result immediately after the examination. This system can identify different area of interest of students and student can be judged with respect to all dimensions of the subject. Reliability of the results improves as all the students are tested on equal predefined footings. Introduction Oral exmaninations have always been considers an important element of the total student evaluation system. They are being used in different situations for different purposes. The assumption on which the use of oral examination has been based is that there are many things which cannot be tested by paper-pencil examination, while they can be tested by oral examinations. However, a group of teachers feel that oral examination provides situation for in-depth questioning, using variety, constructing question based on students response, which give oral exmination a uniqueness not possible in other form of tests. In addition, there are the other skills and abilities like shrpness in comprehending the question, organizing the response, logical presentation of one’s point of view, the way of presenting giving emphesis at right points, etc., which if to be tested then oral examination is the only way. In courses where development of oral communication skills is the main objective e.g. for salesmanship, marketing products orally in one-to-one situation or one-to-large group presentation then oral examination has to be designed and developed accordingly. There are some inherent problems in oral examination – the question of bias, hallo-effect, reliability, variations in sets of questions asked to different students and examiners personal likeness and dislikeness during the interaction between examininers and examinees. It has, therefore, always been prefered to use paper-pencil test, in place oral examination if the objectives are served. Problems in the oral examination Past research has often confirmed the poor reliability and validity of oral examinations. But little is known about the factors influencing the evaluation of orals. Contrary to their importance, oral examinations have been the least researched of all forms of assessment. The social interaction during the examination complicates any analysis of it. Some of the research finding on oral examination are: • Barnes and Pressey (1929) stated the variation of the same candidate’s marks as obtained by different examiners. • Trimble (1934) and two other lecturers rated 25 students of Psychology during an oral examination on eight rating scales. The inter rater reliability for the different pairs of examinars varied from – 0.231 to 0.796 for the different scales. For the general estimate of the relative position among students, correlations of 0.33, 0.40 and 0.26 were found. • Birkel (1978) studied that the inter rater reliability of pairs of examineers who, at short intervals, ask the same candidates questions on the same objective in the same subject was around 0.40 to 0.60. • The correlation between oral and written or practical examination with the same contents range