Multitask Models for Controlling the Complexity of Neural Machine Translation Sweta Agrawal Department of Computer Science University of Maryland sweagraw@cs.umd.edu Marine Carpuat Department of Computer Science University of Maryland marine@cs.umd.edu Abstract We introduce a machine translation task where the output is aimed at audiences of different levels of target language proficiency. We collect a novel dataset of news articles available in English and Spanish and written for diverse reading grade levels. We leverage this dataset to train multitask sequence to sequence models that translate Spanish into English targeted at an easier reading grade level than the original Spanish. We show that multitask models outperform pipeline approaches that translate and simplify text independently. 1 1 Introduction Generating text at the right level of complexity is important to make machine translation (MT) more accessible to non-native speakers, language learners (Petersen and Ostendorf, 2007; Allen, 2009) or people who suffer from language impairments (Carroll et al., 1999; Canning et al., 2000; Inui et al., 2003). Simplification has been used to improve MT by restructuring complex sentences into shorter and simpler segments that are easier to translate (Gerber and Hovy, 1998; ˇ Stajner and Popovic, 2016; Hasler et al., 2017). Closest to our goal, Marchisio et al. (2019) address the task of producing either simple or complex translations of the same input, using automatic readability scoring of parallel corpora. Our work shares their goal of controlling translation complexity, but considers a broader range of reading grade levels and simplification operations grounded in professionally edited text simplification corpora. We collect examples of Spanish sentences paired with several English translations that span a range of complexity levels from the Newsela website, which provides professionally edited simplifications and translations. While Newsela dataset has been used to build English text simplification systems (Xu et al., 2016; Zhang and Lapata, 2017; Scarton and Specia, 2018; Nishihara et al., 2019; Zhong et al., 2020) and Spanish simplification systems ( ˇ Stajner et al., 2018), we exploit the document level alignment between English and Spanish articles to construct evaluation and training samples for complexity controlled MT. By contrast with MT parallel corpora, the English and Spanish translations at different grade levels are only comparable. We adopt a multitask approach that trains a single encoder-decoder model to perform the two distinct tasks of machine translation and text simplification and evaluate it on Spanish- English complexity controlled MT. Our empirical study shows that multitask models produce better and simpler translations than pipelines of independent translation and simplification models. Scripts to replicate our model configurations and our cross-lingual segment aligner are available at https: //github.com/sweta20/ComplexityControlledMT. 2 A Multitask Approach to Complexity Controlled MT We define complexity controlled MT as a task that takes two inputs: an input language segment s i and a target complexity c representing the desired reading grade level of the output. The goal is to generate a translation s o in the output language with complexity c. 1 This paper is an abridged version of our work (Agrawal and Carpuat, 2019). This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/.