Evaluation of a Mixed-Initiative Dialogue Multimodal Interface Baoli Zhao, Tony Allen and Andrzej Bargiela The Nottingham Trent University {baoli.zhao,tony.allen,andrzej.bargiela}@ntu.ac.uk www.ntu.ac.uk Abstract This paper presents the speech recognition accuracy testing and usability evaluation of a mixed-initiative dialogue multimodal interface for the ATTAIN * travel information system [1]. Experimental results show that although the speech recognition accuracy of the interface is less (sample accuracy rate 74.5%) than that of an equivalent directed-dialogue interface (sample accuracy rate 88.5%), the usability is seen to be significantly improved in terms of effectiveness, efficiency, learnability and user satisfaction. 1. Introduction The main objective of this research is to investigate a robust speech-enabled query interface for the ATTAIN travel information system. In recent years, the use of speech and natural language interface technologies have shown great promise for significantly improving the usability of many computer based interactive applications [2]. The creation of a speaker-independent, speech-enabled interface system, especially one that could provide a natural language speech dialogue, was therefore thought to be of great benefit to future users of the ATTAIN system as it would immediately open up the system facilities to these users who are either unable or unwilling to use the existing WAP based interface. To develop the speech-enabled interfaces, Voice eXtensible Markup Language (VoiceXML) [3] is used. This language is based on web standards and can be used to easily create speech-enabled applications. Our previous research has focussed on a creating a directed-dialogue speech- enabled interface for the ATTAIN travel information systems. As a result of this research, a directed-dialogue interface was developed and is presented in [4]. However, the previous usability test results show that the directed-dialogue interface only gave the lowest level of performance that is acceptable to the users. Although speech remains the most natural input method for communication with information systems, the previous interface had many problems in terms of usability performance and user satisfaction. In order to overcome the identifiable difficulties * Advanced Traffic and Travel Information System