The Role of Diacritics in Adapting the Difficulty of Arabic Lexical Recognition Tests Osama Hamed Language Technology Lab University of Duisburg-Essen osama.hamed@uni-due.de Torsten Zesch Language Technology Lab University of Duisburg-Essen torsten.zesch@uni-due.de Abstract Lexical recognition tests are widely used to assess the vocabulary size of language learn- ers. We investigate the role that diacritics play in adapting the difficulty of Arabic lexi- cal recognition tests. For that purpose, we im- plement an NLP pipeline to reliably estimate the frequency of diacritized word forms. We then conduct a user study and compare Arabic lexical recognition tests in three settings: (i) without diacritics, (ii) with the most frequent diacritized form of a root, and (iii) the least fre- quent diacritized form of a root. We find that the use of infrequent diacritics can be used to adapt the difficulty of Arabic lexical recogni- tion tests and to avoid ceiling effects. 1 Introduction Lexical recognition tests (LRTs) are used to mea- sure the vocabulary size of a learner. For that pur- pose, learners are presented with lexical items and have to decide whether they are part of the vocabu- lary of a given language (i.e. a word) or not (i.e. a nonword). Figure 1 gives an example of the two most common presentation formats: (i) Yes/No questions and (ii) checklists. A lexical recognition test consists of a relatively small number of words and nonwords, usually 40 words and 20 nonwords. It has been shown that such a small number of items is sufficient to consistently measure the vo- cabulary size (Huibregtse et al., 2002). As a con- sequence, lexical recognition tests are easy to ad- minister and fast (Lemh¨ ofer and Broersma, 2012). Nonwords in a lexical recognition test are typ- ically used as distractors. Thus, they should be close to existing words and are usually created by swapping letters in existing words (Stubbe, 2012) or by generating character sequences based This work is licensed under a Creative Commons Attri- bution 4.0 International Licence. Licence details: http: //creativecommons.org/licenses/by/4.0/. (a) Yes/No format (b) Checklist format Figure 1: Examples of lexical recognition tests. on position-specific character language models (Hamed and Zesch, 2015). Words in a lexical recognition test have the function to measure the vocabulary size, thus the test needs to contain words from many frequency bands, i.e. very fre- quent words like door or large as well as less com- mon words like obey or forfeit. While lexical recognition tests are well- established for English (Lemh¨ ofer and Broersma, 2012), and other European languages like Ger- man and Dutch (Lemh¨ ofer and Broersma, 2012), French (Brysbaert, 2013) and Spanish (Izura et al., 2014), there is still very little work on Arabic LRTs. The studies by Baharudin et al. (2014) and Ricks (2015) neglect lexical diacritics, a very important feature of the Arabic language that causes many challenges for automatic processing (Farghaly and Shaalan, 2009). The Arabic script contains two classes of symbols: letters and diacritics (Habash, 2010). Whereas letters are always written, diacritics are optional. Diacritics are usually used in specific settings like language teaching or religious texts. This leads to a high amount of ambiguity of a non- diacritized Arabic word. Figure 2 compares the Osama Hamed and Torsten Zesch 2018. The role of diacritics in increasing the difficulty of Arabic lexical recognition tests. Proceedings of the 7th Workshop on NLP for Computer Assisted Language Learning at SLTC 2018 (NLP4CALL 2018). Linköping Electronic Conference Proceedings 152: 23–31. 23