A Simple Transfer Learning Baseline for Ellipsis Resolution Rahul Aralikatte 1 , Matthew Lamm 1,2 , Daniel Hardt 3 , and Anders Søgaard 1 1 University of Copenhagen, 2 Stanford University 3 Copenhagen Business School Abstract Most, if not all, forms of ellipsis (e.g., ‘so does Mary’) are similar to reading compre- hension questions (‘what does Mary do’), in that in order to resolve them, we need to identify an appropriate text span in the pre- ceding discourse. We present a strong base- line for English ellipsis resolution that exploits this similarity by relying on architectures de- veloped for machine reading comprehension. We present both single-task transfer learning models and joint models, trained on machine reading comprehension and coreference reso- lution datasets, clearly outperforming the cur- rent (from 2016-18) state of the art for Sluice Ellipsis (from 0.67 to 0.86 F 1 ) and Verb Phrase Ellipsis (from 0.65 to 0.79 F 1 ). 1 Introduction Ellipsis resolution is a hard, open problem in NLP, and an important source of error in machine trans- lation, question answering, and dialogue under- standing (Vicedo and Ferrandez, 2000; Dzikovska et al., 2006; Chung and Gildea, 2010; Macketanz et al., 2018; Hansen and Søgaard, 2020). There are no large annotated text corpora for this phe- nomenon, even for English, and we only have an- notations for a subset of the known ellipsis con- structions. Since annotation is expensive and cum- bersome, any synergies with existing NLP tasks could be useful and enable us to leverage auxiliary data sources when learning models for ellipsis res- olution. This paper presents a simple baseline approach to ellipsis resolution based on a straightforward observation, depicted in Figure 1. Ellipsis reso- lution can be converted to a machine reading com- prehension (MRC) problem. The same holds for coreference resolution. Ellipsis, coreference, and questions put in focus referentially dependent ex- pressions (Carlson, 2006), or free variables (Par- tee, 1978), that need to be resolved in order to “I can’t remember where, but my wife is waiting for me”, Mr. Smith said. Ellipsis → QA Q: What can’t Mr. Smith remember? A: Where his wife is waiting Figure 1: Sluice Ellipsis can be reformulated as ma- chine reading comprehension. Similarly, for coref- erence resolution, we can ask: Who can’t remember where his wife is waiting? comprehend the discourse. For similar obser- vations about different tasks, see McCann et al. (2018a); Gardner et al. (2019). This straightforward observation leads us to suggest treating different forms of ellipsis resolu- tion – and, later, as an auxiliary task, also corefer- ence resolution – as an MRC problem, and to ap- ply state-of-the-art architectures for MRC to ellip- sis resolution tasks, as well as to experiment with using training data for MRC and coreference reso- lution to improve our new baseline for ellipsis res- olution Contributions We cast ellipsis and coreference as MRC problems, enabling us to induce mod- els for these tasks using neural architectures orig- inally developed for MRC. Applying these archi- tectures out of the box enables us to establish a strong baseline for ellipsis resolution tasks, im- proving significantly over previous work. Using the same architecture for the different ellipsis res- olution tasks, as well as for MRC and corefer- ence resolution, enables us to explore synergies between the tasks, and we show that training joint models for multiple tasks leads to even better per- formance. arXiv:1908.11141v2 [cs.CL] 15 Apr 2020