The Effect of Entity Recognition on Answer Validation ´ Alvaro Rodrigo, Anselmo Pe˜ nas, Jes ´ us Herrera, Felisa Verdejo Departamento de Lenguajes y Sistemas Inform´ aticos Universidad Nacional de Educaci´ on a Distancia Madrid, Spain {alvarory, anselmo, jesus.herrera, felisa}@lsi.uned.es Abstract. The Answer Validation Exercise (AVE) 2006 is aimed at evaluating systems able to decide whether the responses of a Question Answering (QA) sys- tem are correct or not. Since most of the questions and answers contain entities, the use of a textual entailment relation between entities is studied here for the task of Answer Validation. We present some experiments concluding that the en- tity entailment relation is a feature that improves a SVM based classifier close to the best result in AVE 2006. 1 Introduction The Answer Validation Exercise (AVE) [8] of the Cross Language Evaluation Forum (CLEF) 2006 is aimed at developing systems able to decide whether the responses of a Question Answering (QA) system are correct or not using a Textual Entailment ap- proach [3]. The AVE 2006 test corpus contains hypothesis-text pairs where the hypothe- ses are built from the questions and answers of the real QA systems, and the texts are the snippets given by the systems to support their answers. Participant systems in AVE 2006 must return a value YES or NO for each hypothesis-text pair to indicate if the text entails the hypothesis or not (i.e. the answer is correct according to the text). The questions and answers of the QA Track at CLEF contain many entities (person names, locations, numbers, dates...) due to the nature of the questions in this evaluation: 75% of the questions in Spanish were factoids in the previous campaign [10]. For this reason, we studied the consideration of an entity entailment relation in combination with a SVM classifier to solve the Answer Validation problem. Section 2 describes the process of entity detection and entailment decision. Section 3 describes the experimental setting combining the entity based entailment decision with a baseline SVM system. Section 4 shows the results of the experiments together with an error analysis. Finally, we give some conclusions and future work in Section 5. 2 Entailment between entities The first step for considering an entailment relation between entities is to detect them in a robust way that minimizes the effect of errors in the annotation.