A Speech-to-Speech, Machine Translation Mediated Map Task: An Exploratory Study Loredana Cerrato 1 , Hayakawa Akira 1 , Nick Campbell 1 , and Saturnino Luz 2 1 ADAPT Centre School of Computer Science and Statistics Trinity College Dublin, Ireland {cerratol,campbeak,nick}@tcd.ie 2 Usher Institute of Population Health Sciences & Informatics University of Edinburgh, UK S.Luz@ed.ac.uk Abstract. The aim of this study is to investigate how the language tech- nologies of Automatic Speech Recognition (ASR), Machine Translation (MT), and Text To Speech (TTS) synthesis affect users during an inter- lingual interaction. In this paper, we describe the prototype system used for the data collection, we give details of the collected data and report the results of a usability test run to assess how the users of the inter- lingual system evaluate the interactions in a collaborative map task. We use widely adopted usability evaluation measures: ease of use, effective- ness and users satisfaction, and look at both qualitative and quantitative measures. Results indicate that both users taking part in the dialogues (instructions giver and follower) found the system similarly satisfactory in terms of ease of learning, ease of use, and pleasantness, even if they were less satisfied with its effectiveness in supporting the task. Users employed different strategies in order to adapt to the shortcomings of the technology, such as hyper-articulation, and rewording of utterances in relation to error of the ASR. We also report the results of a com- parison of the map task in two different settings – one that includes a constant video stream (“video-on”) and one that does not (“no-video.”) Surprisingly, users rated the no-video setting consistently better. Keywords: Interlingual Speech-to-Speech Translation, Repair Strate- gies, Speaker Alignment, Adaptation, User Evaluation. 1 Introduction Speech-to-Speech (S2S) translation systems are becoming a daily reality as a way of communicating. Recently Microsoft announced and publicly demonstrated the Skype Translator: a system that enables cross-lingual conversations in real time. Although automatic S2S translation systems are in commercial deployment they are still not adapted to the way in which people actually behave when using them. To fill this gap we conducted an exploratory study using a prototype system to collect a set of 15 interactions in a S2S multimodal interlingual setting.