Vol.8 (2018) No. 4-2 ISSN: 2088-5334 Question Classification Based on Bloom’s Taxonomy Using Enhanced TF-IDF Manal Mohammed # , Nazlia Omar # # Faculty of Information Science and Technology, Center for Artificial Intelligence Technology, Universiti Kebangsaan Malaysia, Bangi 43600, Malaysia E-mail: manal.altamimi@outlook.com, nazlia@ukm.edu.my Abstract— Bloom’s Taxonomy has been used widely in the educational environment to measure, evaluate and write high-quality exams. Therefore, many researchers have worked on the automation for classification of exam questions based on Bloom’s Taxonomy. The aim of this study is to make an enhancement for one of the most popular statistical feature, which is TF-IDF, to improve the performance of exam question classification in accordance to Bloom’s Taxonomy cognitive domain. Verbs play an important role in determining the level of a question in Bloom’s Taxonomy. Thus, the improved method assigns the impact factor for the words by taking the advantage of the part-of-speech tagger. The higher impact factor assigns to the verbs, then to the noun and adjective, after that, the lower impact factor assigns to the other part-of-speech. The dataset that has been used in this study is consist of 600 questions, divided evenly into each Bloom level. The questions first pass into the preprocessing phase in which they are prepared to be suitable for applying the proposed enhanced feature. For classification purpose, three machine learning classifiers are used Support Vector Machine, Naïve Bayes, and K-Nearest Neighbour. The enhanced feature shows satisfactory result by outperforming the classical feature TF-IDF via all classifiers in terms of weighted recall, precision, and F1-measure. On the other hand, Support Vector Machine has superior performance over other classifiers Naïve Bayes, and K-Nearest Neighbour by achieving an average of 86%, 85%, and 81.6% weighted F1-measure respectively. However, these results are promising and encouraging for further investigations. Keywords— question classification; bloom’s taxonomy; TF-IDF; support vector machine; naïve bayes; K-Nearest Neighbour. I. INTRODUCTION The most traditional and classical way to evaluate students in educational institutes is by written examination. Therefore, many lecturers are trying to follow some framework such as Bloom’s Taxonomy while preparing the exam questions to ensure the production of high-quality exams. The benefits of classifying questions regarding Bloom’s Taxonomy Cognitive Domain (BTCD) are providing a suitable and appropriate way to measure students’ intellectual abilities [1], and covering different thinking skills start from simplest to the most complex one. Therefore, the automatic classification of examination questions based on Bloom’s taxonomy is highly required, especially in the educational environment [2], since the process of classifying exam questions manually is time- consuming. Furthermore, some academicians have no idea about Bloom’s taxonomy [3], or have no ability to distinguish the difference between Bloom’s taxonomy's levels which may lead to misclassification. Hence, this may lead to poor quality examinations [3][4]. Benjamin Bloom and his team introduced Bloom’s taxonomy in 1956 which basically involves three domains. The domain that has developed for the purpose of assessing students’ intellectual abilities and skills is known as Cognitive Domain [1]. Cognitive domain has a hierarchical structure which comprises six levels namely knowledge level, comprehension level, application level, analysis level, synthesis level and evaluation level. Knowledge level evaluates students’ ability in memorizing facts and basic information e.g. Label the parts of the microscope shown on the right. Comprehension level measures students’ ability in understanding ideas and topics based on previous knowledge e.g. Describe in your own words what happens when a stream's velocity slows. Application level evaluates students’ skills in implementing acquired knowledge to new circumstances e.g. Apply the storytelling technique here to a little story of your own. Analysis level assesses students’ ability in dividing information into pieces to classify them and find the relationship e.g. Break down the main actions of the story. Synthesis level evaluates students’ ability to combine ideas together to create new solution e.g. Create a set of guidelines to determine the points of a plant susceptible to localized corrosion. Evaluation level measures students’ ability to defend and judge issues based on some criteria e.g. Assess the relative effectiveness of different 1679 brought to you by CORE View metadata, citation and similar papers at core.ac.uk provided by International Journal on Advanced Science, Engineering and Information Technology