Tracing Translations in the Making Elliot MACKLOVITCH Ngoc Tran NGUYEN RALI - DIRO Universit´ e de Montr´ eal Montr´ eal, Qu´ ebec, Canada, H3C 3J7 {macklovi,nguyennt,lapalme}@iro.umontreal.ca Guy LAPALME Abstract This paper presents TTPlayer, a trace file analysis tool used to develop TransType, an innovative computer-aided translation system. We first discuss the context of the project and the design of the tracing tool. We show how it was used for discovering interesting patterns of use as well to guide further developments in the TT2 project. 1 Introduction In the world of sports, it is common practice to film games or competitions in order to analyse the tactics of an opposing team, or simply to improve one’s own performance. The ability to stop, slow down and replay the action often al- lows one to detect things that would otherwise escape attention: a telltale gesture a pitcher makes just before throwing to first base; a ten- dency to lift the head early and follow the ball, thereby making one’s drive erratic. Now wouldn’t it be nice if we could do the same thing in other areas of human activity? In the sciences, for example, where the aim is often to achieve a better understanding of the behaviour of a human expert so that we can pro- vide her with an intelligent form of automated assistance. Alas, more often than not, it turns out that this is just not possible either for prac- tical reasons or for ethical ones. With the excep- tion of professional athletes, artistic perform- ers and exhibitionists, people do not generally like to be closely observed at their work for ex- tended periods of time; it riles them and tends to make them uneasy. Nor do personal inter- views in which human experts are asked to ex- plicitly describe their work practices or thought processes offer a satisfactory alternative. For one thing, the expert’s answers are often un- reliable, because she may not be aware of, or be able to make explicit what they do instinc- tively; for another, the expert may provide the responses she thinks the interviewer wants to hear. Whence the interest of an automatic trac- ing program, particularly in the context of an interactive computer application. The trac- ing program doesn’t overtly observe the ex- pert, thereby ensuring that her tasks will be performed naturally. Instead, it unobtrusively records the expert’s every action as she works with the system, as well as the system’s re- sponse to each such action, and discreetly stores them in an electronic file that can later be an- alyzed in detail. Just as with the filming of a sports event, careful analysis of this trace file can often prove very revealing; or at least this has been our experience. We have designed such a tracing facility as part of our contribution to a research project called TransType, the goal of which is to explore the feasibility of a new type interactive machine translation (Foster et al., 1997; Langlais et al., 2002a; Langlais et al., 2001). Our automatic tracing program, chris- tened TTPlayer, analyses the trace file of a translator’s working session using TransType and generates detailed statistics on a host of in- teresting questions. Furthermore, TTPlayer can also read the trace file and visually replay the translation session, much like a video cas- sette recorder. In effect, it’s almost as though we were actually present and able to peer over the translator’s shoulder, except that we can stop, slow down, accelerate and rewind the ac- tion at will. This too facilitates our efforts to better understand the behaviour and strategies of the expert user, in this case a professional translator. The remainder of this paper is structured as follows. In the next section, we will provide a general overview of the TransType system. In section 3, we will then describe TTPlayer in some detail: not just how the program func- tions, but also its role in the usability trials that were central to the TT2 project. We conclude 323