Where is Linked Data in Question Answering over Linked Data? Tommaso Soru 12 , Edgard Marx 13 , Andr´ e Valdestilhas 2 , Diego Moussallem 2 , Gustavo Publio 2 , and Muhammad Saleem 2 1 Liber AI Research, London, UK 2 AKSW, University of Leipzig, Germany 3 HTWK Leipzig, Germany tom@tommaso-soru.it Abstract. We argue that “Question Answering with Knowledge Base” and “Question Answering over Linked Data” are currently two instances of the same problem, despite one explicitly declares to deal with Linked Data. We point out the lack of existing methods to evaluate question answering on datasets which exploit external links to the rest of the cloud or share common schema. To this end, we propose the creation of new evaluation settings to leverage the advantages of the Semantic Web to achieve AI-complete question answering. 1 Introduction Question Answering with Knowledge Base (Kbqa) parses a natural-language question and returns an appropriate answer that can be found in a Knowledge Base (KB). Currently, one of the most exciting scenarios for Question Answer- ing (QA) is the Web of Data, a fast-growing distributed cloud of interlinked KBs which comprises more than 100 billions of edges [13]. Similarly, Question Answering over Linked Data (Qald) is a research field aimed at transforming utterances into Sparql queries which can be executed towards the Linked Open Data (Lod) cloud [11]. Qald and Kbqa are strictly related, as they both tar- get the retrieval of answers from KBs. However, the current benchmarks and datasets available for evaluating Qald approaches are limited to an unlinked and unstandardized vision of the structured question answering task. In this position paper, we point out the lack of existing methods to evaluate QA on datasets which exploit external links to the rest of the cloud. Moreover, we ar- gue that several learning-based Kbqa approaches may be very competitive in Qald challenges, as the current distinctions among their respective benchmarks are only in terms of underlying KBs. Instead, our plea is to let language experts do language and Web semantics experts do semantics. We propose the creation of new evaluation methods and settings to leverage the advantages of the Semantic Web (SW) to achieve AI-complete QA over the Web of Data [6]. arXiv:2005.03640v1 [cs.CL] 7 May 2020