Spoken and Multimodal Bus Timetable Systems: Design, Development and Evaluation Markku Turunen, Jaakko Hakulinen, Esa-Pekka Salonen, Anssi Kainulainen and Leena Helin Speech-based and Pervasive Interaction group, TAUCHI, Department of Computer Sciences University of Tampere, Tampere, Finland {Markku.Turunen,Jaakko.Hakulinen,Esa-Pekka.Salonen, Anssi.Kainulainen, Leena.Helin}@cs.uta.fi The systems have been developed in collaboration with various universities and companies. Stopman has been avail- able to public since 2003. The systems are based on the common Jaspis architecture to be presented next. Abstract We present three speech-based bus timetable systems with different approaches. The first system had open user-initiative dialogues aiming at natural interaction. The user tests re- vealed several problems with this approach, and we devel- oped solutions from multiple perspectives to overcome the problems. The second system focuses on task-oriented sys- tem-initiative dialogues, and contains a multimodal interface for smart mobile terminals. The third has a flexible mixed- initiative dialogue strategy, and offers multimodal integrated tutoring. All systems share a common agent-based architec- ture. We discuss solutions for design, development and evaluation of this kind of information systems. 2. Common system architecture The timetable systems are built on top of the Jaspis architec- ture [2]. Jaspis is based on the agents – managers – evaluators –paradigm, where managers coordinate sets of evaluators to select appropriate agents that actually handle the tasks (e.g., dialogue decisions). Agents in Jaspis-based applications are compact software components. Managers represent the high level organization of tasks that are handled within spoken dialogue applications. All components share all the informa- tion via Information Storage. 1. Introduction Figure 1 illustrates the components of the timetable sys- tems. All three bus timetable applications use this high-level structure. The systems contain standard Jaspis components (Communication, Dialogue, Presentation and Input manag- ers), and an additional manager, the Database Manager (Task Manager in Interact), which is used to communicate with the timetable database. We have developed multiple bus timetable systems in various research projects. The timetable domain is practical, provides plenty of research challenges, and the results can be used in other similar information systems. Timetable services are suitable also for mobile and speech-based interfaces. Existing human-operated systems can be replaced with automatic sys- tems, and new services can be produced for areas that are not possible or economically viable with human operators. These automated systems might be the only possibility for such special user groups as visually impaired people to access certain information. Presentation Manager Communication Manager Input Manager Dialogue Manager Interaction Manager (facilitator) Jaspis Core Information Manager Database/Task Manager Input Agents Input Evaluators Presentation Agents Presentation Evaluators IO- Agents IO- Evaluators Dialogue Agents Dialogue Evaluators Database Agents Database Evaluators Timetable Database Information Storage Domain and Concept Model We present solutions for spoken dialogue system devel- opment challenges based on the experiences from the iterative design, implementation and evaluation of Interact, Stopman and Busman systems. The functionality of Interact and Bus- man is similar to other speech-based timetable services, such as MALIN [1]. Users may query information such as bus routes and their timetables. Stopman serves bus stop specific timetables. All systems can be used with mobile and fixed line telephones. Example 1 demonstrates dialogues with the systems. Figure 1: General system architecture. The differences in the systems are in the structure of the Information Storage (modeling of dialogue and domain in- formation), and in the agents that use that information to carry out interaction tasks. Next, the features of each system are presented. We go through the systems starting from the Inter- act system, after which the Stopman and the Busman systems, and their multimodal extensions are presented. Each system has different user interface, and uses differ- ent approach to dialogue management. During the develop- ment of these systems we have faced several challenges and developed solutions to these using agent-based technologies. In the user tests of the Interact system we found that open user-initiative dialogue strategy based on human-human in- teraction failed to provide enough information to users and take the initiative when needed. The Stopman system pro- vides a task-oriented, system-initiated interface while the Busman system aims for truly mixed-initiative dialogue man- agement. We have also developed multimodal extensions to the Busman and the Stopman systems to provide guidance and graphical elements to the interface. 3. Natural interaction - the Interact system The main purpose of the Interact project was to investigate natural interaction methods. Development took advantage of a corpus recorded from real calls to the human operated Hel- sinki local traffic information service. Adaptivity was the