Zero-shot Multi-lingual Interrogative Qestion Generation for “People Also Ask” at Bing Rajarshee Mitra, Rhea Jain, Aditya Srikanth Veerubhotla, Manish Gupta {ramitra,rhea.jain,aditya.veerubhotla,gmanish}@microsoft.com Microsof India ABSTRACT Multi-lingual question generation (QG) is the task of generating natural language questions for single answer passage in any given language. In this paper, we design a system for supporting multi- lingual QG in the “People Also Ask” (PAA) module for Bing. For zero shot seting, the primary challenge is to transfer the knowl- edge from trained QG model in the pivot language to other lan- guages without further addition of training data in these languages. Compared to other zero-shot tasks, the diferentiating and chal- lenging aspect in QG is to preserve the question structure so that the resulting output is interrogative. Existing models for similar tasks tend to generate natural language queries or copy sub-span of the passage, failing to preserve the question structure. In our work, we demonstrate how knowledge transfer in multi-lingual IQG (Interrogative QG) can be signifcantly improved using aux- iliary tasks either in multi-task or pre-training task seting. We explore two kinds of tasks – cross-lingual translation and multi- lingual denoising auto-encoding of questions, especially when us- ing translate-train. Using data for 13 languages from Bing PAA as well as online A/B tests, we show that both of these tasks signif- icantly improve the quality of zero-shot IQG on non-trained lan- guages. CCS CONCEPTS • Computing methodologies → Neural networks; Natural lan- guage generation;• Information systems → Web search en- gines; Content ranking. KEYWORDS interrogative question generation, deep learning, natural language processing, zero shot modeling, multi-lingual denoising auto-encoding, People Also Ask ACM Reference Format: Rajarshee Mitra, Rhea Jain, Aditya Srikanth Veerubhotla, Manish Gupta. 2021. Zero-shot Multi-lingual Interrogative Qestion Generation for “Peo- ple Also Ask” at Bing. In Proceedings of the 27th ACM SIGKDD Confer- ence on Knowledge Discovery and Data Mining (KDD ’21), August 14–18, Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proft or commercial advantage and that copies bear this notice and the full cita- tion on the frst page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permited. To copy otherwise, or re- publish, to post on servers or to redistribute to lists, requires prior specifc permission and/or a fee. Request permissions from permissions@acm.org. KDD ’21, August 14–18, 2021, Virtual Event, Singapore © 2021 Association for Computing Machinery. ACM ISBN 978-1-4503-8332-5/21/08…$15.00 https://doi.org/10.1145/3447548.3469403 Q: deep learning framework Q: francia cultura y tradiciones (A) English (B) Spanish Figure 1: Examples of the “People Also Ask” module from Bing (retrieved 8-Feb-2021) 2021, Virtual Event, Singapore. ACM, New York, NY, USA, 9 pages. https: //doi.org/10.1145/3447548.3469403 1 INTRODUCTION Qestion generation (QG) involves producing natural language ques- tion from a passage, ofen accompanied by the actual answer to the question. QG is very useful in information retrieval spanning from question answering, passage ranking, machine reading com- prehension to conversational models. It can help in creating more question-passage mappings that can lead to beter ranking. For learning management systems, it could be useful for automatically designing quiz questions. In question answering, it could be useful in creating an ofine corpus of QA pairs beforehand for fast lookup. More recently, it has been shown that adding generated questions to passage helps in improved passage ranking [26]. Other uses in- clude creating FAQs automatically from unstructured content like product descriptions or processes. Te most widely known form of QG is answer-aware QG where the generated question relates to a specifc answer from within a big passage [25, 35, 36, 40]. Another form of QG involves gener- ating free form QG from passages [6, 7, 30, 34, 38] without any extra answer input. Commercial search engines ofen produce de- scriptive passage answer (or featured snippets) to user queries (e.g., “why did usa bomb japan”, “what kind of houses eskimos live in”). Here, QG plays a pivotal role in creating QA corpus that can be used for retrieval. In most setings like designing quiz questions, creating questions for QA systems, generating questions for Peo- ple Also Ask module 1 , we need well-formed questions and not key- word queries. We show examples of PAA module in English as well as Spanish in Fig. 1. “People Also Ask” module allows users to discover related questions pertaining to the current query. Such a module can be useful in (1) reducing gap between user’s infor- mation need and the query, especially for non-expert users, (2) 1 htps://www.searchenginewatch.com/2020/10/14/people-also-ask-paa-feature- uncovering-googles-hidden-gem/ ADS Track Paper KDD ’21, August 14–18, 2021, Virtual Event, Singapore 3414