Augmenting Named Entity Recognition with Commonsense Knowledge Ghaith Dekhili, Ngoc Tan Le, Fatiha Sadat University of Quebec in Montreal 201 President Kennedy avenue, H2X 3Y7 Montreal, Quebec, Canada dekhili.ghaith, le.ngoc tan, sadat.fatiha@uqam.ca Abstract Commonsense can be vital in some applications like Natural Language Understanding (NLU), where it is often required to resolve ambiguity arising from implicit knowledge and under- specification. In spite of the remarkable success of neural network approaches on a variety of Natural Language Processing tasks, many of them struggle to react effectively in cases that re- quire commonsense knowledge. In the present research, we take advantage of the availability of the open multilingual knowledge graph ConceptNet, by using it as an additional external re- source in Named Entity Recognition (NER). Our proposed architecture involves BiLSTM layers combined with a CRF layer that was augmented with some features such as pre-trained word embedding layers and dropout layers. Moreover, apart from using word representations, we used also character-based representation to capture the morphological and the orthographic informa- tion. Our experiments and evaluations showed an improvement in the overall performance with +2.86 in the F1-measure. Commonsense reasonnig has been employed in other studies and NLP tasks but to the best of our knowledge, there is no study relating the integration of a commonsense knowledge base in NER. 1 Introduction NLP and Machine Learning (ML) communities have long been interested in developing models capable of commonsense reasoning. In addition to that, commonsense can be vital in some applications like NLU, where it is often required to resolve ambiguity arising from implicit knowledge and under-specification by taking word meaning and context into account (Havasi et al., 2010). ConceptNet (Speer et al., 2016) is a knowledge graph designed to represent the general knowledge involved in understanding languages to improve natural language applications. When word embed- dings extracted from ConceptNet, which represent relational knowledge (ConceptNet PPMI (Speer et al., 2016)), are combined with word embeddings acquired from distributional semantics such as Word2Vec (Mikolov et al., 2013), they provide applications with understanding that they would not acquire from distributional semantics alone, nor from narrower resources such as WordNet or DBPedia. 2 Methodology The present research aims at integrating commonsense knowledge into a Named Entity Recognition (NER) task in order to learn more entities and improve the efficiency and effectiveness of the NER. This research relies on the work presented by Speer et al.(2016), who created a robust set of embed- dings (ConceptNet Numberbatch (Speer et al., 2016)), that represents both ConceptNet and distributional word embeddings learned from text. This set of embeddings represents different domains and has com- plementary strengths. As in Lample et al. (2016), we used in our architecture a Bi-directionnal Long Short-Term Memory (BiLSTM) layers, combined with a Conditional Random Field (CRF) layer (Laf- ferty et al., 2001), augmented with some features such as pretrained word embedding layers and dropout layers. Apart from using word representations, we used also character-based representation to capture This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/.