A Controlled Experiment on the Effects of Machine Translation in Multilingual Requirements Meetings Fabio Calefato, Filippo Lanubile Dipartimento di Informatica Università degli Studi di Bari “A. Moro” Bari, Italy calefato,lanubile@di.uniba.it Rafael Prikladnicki Pontifícia Universidade Católica do Rio Grande do Sul PUCRS Porto Alegre, Brazil rafael.prikladnicki@pucrs.br Abstract—Requirements engineering is a communication- intensive activity and thus it suffers much from language difficulties in global software projects. Remote requirements meetings can benefit from machine translation as this technology is today available in the form of cross-language chat services. In this paper, we present the design of a controlled experiment to investigate the effects of automatic machine translation services in requirements meetings. Experiment participants, using either Italian or Portuguese as native language, are asked to interact with a communication tool from a distance in order to prioritize and estimate requirements. First results show that real-time machine translation is not disruptive of the conversation flow and is accepted with favor by participants. However, concrete effects are expected to emerge when language barriers are critical. Keywords-machine translation; language barrier; requirements engineering; empirical study. I. INTRODUCTION Requirements engineering is a communication-intensive activity and thus it suffers much from language difficulties in global software projects [11], [12], [24]. Language is indeed an important factor that largely accounts for the success of offshore IT work in countries with strong English language capabilities, such as Ireland, the Philippines, India, and Singapore [8], [17]. However, there are several other countries, considered followers in global competition, which are increasing their presence in the global IT market. Brazil is one real example of this situation [9]. Brazil’s IT industry is large – A.T. Kearney consultancy estimates that the sector employs 1.7 million people, including programmers, systems analysts, and managers [20] – and it is growing by 6.5% a year on average since 2005 [4], although the vast majority of the IT companies are focused on domestic clients and do not export. For those who export, US companies are the main clients, accounting for over 80% of demand, followed by Latin America (especially Argentina, Chile, Colombia and Mexico), and Europe (especially Germany, Spain, France, England and Portugal). In this scenario, English language is required as a skill for every person working in the global market. Unfortunately, A.T. Kearney estimates that Brazil has only 10.2 million of English speakers, or 5.4% of the population. Chile, for example, has 34.7% of English speakers; India has 8.2% (which represents 90.6 million). Another study published by KPMG in 2009 indicated that one of the disadvantages of Latin American countries is the lack of English speaking professionals [21]. In this context, there are several initiatives going on, for example, in order to include English in the qualification of the IT professionals in Brazil [9]. However, this may be not enough and, to stay competitive in the global IT market these countries we will have to search for alternative solutions. For this reason, distributed project meetings, such as requirements workshops, can benefit from machine translation, as this technology is today available in the form of cross-language chat services and it might be used in countries, such as Brazil, where there are at the same time opportunities for global projects and the lack of English speaking professionals. In our previous work [6] we run a simulated study to evaluate the feasibility of adopting an automatic, cross- language translation to communication-intensive activities, such as distributed requirements engineering. Although our work proved that state-of-the-art machine translation services could be embedded into synchronous text-based chat with a negligible extra time, being only a simulation, the study did not allow us to hypothesize whether the quality of machine translation services would be good enough to allow participants to complete a complex group task while communicating with their own native language. To further our research, in this paper we investigate by means of an experiment how real-time machine translation can be effectively used during distributed requirements engineering meetings involving multilingual groups. Thus, we propose the following research questions for study: RQ1 – Can machine translation services be used in distributed multilingual requirements meetings, instead of English? RQ2 – How does the adoption of machine translation affect group interaction in distributed multilingual requirements meetings, as compared to the use of English? The remainder of this paper is structured as follows. In Section 2 we briefly overview the state of the art in machine translation services. Section 3 presents the controlled experiment in detail, whereas early results are presented in