Texila International Journal of Academic Research ISSN: 2520-3088 DOI: 10.21522/TIJAR.2014.10.02.Art001 Received: 07-02-2023 Accepted: 20-02-2023 Published on: 28.04.2023 Corresponding Author: nathanmanzambi@yahoo.fr Light RAT-SQL: A RAT-SQL with More Abstraction and Less Embedding of Pre-existing Relations Nathan Manzambi Ndongala Ph.D, Department of Computer Science, Texila American University, Guyana Abstract RAT-SQL is among the popular framework used in the Text-To-SQL challenges for jointly encoding the database relations and questions in a way to improve the semantic parser. In this work, we propose a light version of the RAT-SQL where we dramatically reduced the number of the preexisting relations from 55 to 7 (Light RAT-SQL-7) while preserving the same parsing accuracy. To ensure the effectiveness of our approach, we trained a Light RAT-SQL-2, (with 2 embeddings) to show that there is a statistically significant difference between RAT-SQL and Light RAT-SQL-2 while Light RAT-SQL- 7 can compete with RAT-SQL. Keywords: Deep learning, Natural Language Processing, Neural Semantic Parsing, Relation Aware Transformer, RAT-SQL, Text-To-SQL, Transformer. Introduction The RAT-SQL [1] has been used in Text-to- SQL [2-4] as an encoder transformer. The RAT- SQL framework jointly encodes the question and the schema database to improve the generalization even in unseen databases by the model during the training process. RAT-SQL is based on the relation-aware self-attention mechanism, and address schema encoding and schema linking within a text-to-SQL encoder. The core of RAT-SQL is the abstract pre- existing relation between input tokens. The RAT-SQL model implementation has been trained with more than 50 embedding relation types. The management of relations in the Relation Attention Transformer is challenging: Having more relations can lead the model to capture noise and having fewer relations, the model can miss another important relation trend in data. Previous methods, [2] empirically noticed when injecting syntactic dependency in the graph of RAT-SQL, that having many relations can lead to overfitting. Another insight about pre-existing relations is when a pre-trained language model as BERT [5, 6] is used to enhance RAT-SQL the name-based schema linking (NBSL) become marginal [3] but neither method explicitly assess the acceptable threshold of the number of relations to take into account in a Transformer with pre-existing relations. In this work, we attempt to respond to the following questions: 1. How can we reduce the number of “ pre- existing relations” while preserving high- quality parsing? 2. To what extent does exact match accuracy depend on pre-existing abstract relations in Relation-Aware Transformer / RAT-SQL? First, we constrain the number of relations to be equal to the number of heads of the model. Each head tends to specialize [7], and we hypothesize then each relation will be learned by one head of the model. Secondly, we present a new structure of a relation graph inspired by RAT-SQL [1], SS 2 SQL [2], and database theory [8]. 1