101 4-6 September 2002, Edinburgh, UK, Pages 101-107. Bos, Foster & Matheson (eds): Proceedings of the sixth workshop on the semantics and pragmatics of dialogue (EDILOG 2002), 101 4-6 September 2002, Edinburgh, UK, Pages 101-107. Bos, Foster & Matheson (eds): Proceedings of the sixth workshop on the semantics and pragmatics of dialogue (EDILOG 2002), 101 4-6 September 2002, Edinburgh, UK, Pages 101-107. Bos, Foster & Matheson (eds): Proceedings of the sixth workshop on the semantics and pragmatics of dialogue (EDILOG 2002), 101 4-6 September 2002, Edinburgh, UK, Pages 101-107. Bos, Foster & Matheson (eds): Proceedings of the sixth workshop on the semantics and pragmatics of dialogue (EDILOG 2002), Making Sense of Partial * Markus L¨ ockelt and Tilman Becker and Norbert Pfleger and Jan Alexandersson DFKI GmbH, Stuhlsatzenhausweg 3, D-66123 Saarbr¨ ucken, Germany {loeckelt|becker|pfleger|janal}@dfki.de Abstract We address the challenging task of processing partial multimodal utterances in a mixed-initiative dialogue system for multiple applications such as information seeking, device control etc. Based on four different types of expectations computed by our action plan- ner (aka dialog manager), we distinguish between expected utterances, various degrees of unexpected but plausible utterances, and uninterpretable utter- ances. We include a description of the backbone ar- chitecture and the representation formalisms used. 1 Introduction An important characteristic of mixed-initiative di- alogue are partial utterances that can only be in- terpreted in the context of the previous dialogue. The setting in which we investigate the interpreta- tion of partial utterances is the multimodal, mixed- initiative dialogue system SmartKom. However, it is our larger goal to develop a core dialog back-bone. In fact, some of the modules described here have al- ready been used in other projects. In SmartKom, communicative actions include spoken utterances and two-dimensional gestures (pointing, encircling etc.) by the user and also by an animated presen- tation agent. Since the analyses of both modali- ties are integrated into a common representation, our processing mechanisms are actually modality- independent. In the following we will use utterance to include communicative actions in all modalities. There are a number of phenomena that occur nat- urally in such dialogues and systems, including the need for robustness against fairly common recogni- tion errors. In this paper, we focus on partial utter- ances: Those in the context of user-initiative, usu- ally additions to or changes of the current discourse context and those in the context of user-response, usually a reaction to a system request. This paper first presents the dialog system SmartKom, its discourse and domain representa- tion language and the data flow of processing. Sec- * The research within SmartKom presented here is funded by the German Ministry of Research and Technology under grant 01 IL 905. tion 2.2 presents the discourse modeler and 2.3 goes into more detail about planning a system action and supplying expectations for the next user utterance. The central sections are 3 and 4 which describe the details of interpretation as the integration of partial utterances into a coherent dialog context. Before we conclude our paper we discuss and compare our approach in the light of QUD in section 5. 2 Architecture Our back-bone is divided into different modules (see figure 1), each specialized on a different task. A multi-blackboard architecture (see, e.g., (Wahlster, 2000)) is used for communication where different modules publish or subscribe to indicate read or write permission for so-called data pools. All mod- ules within the back-bone exchange information us- ing a common representation: the domain model (see below). For the analysis part of the back-bone, either a single application object or one or more sub- objects are wrapped into a hypothesis, containing additional information, like syntactic information, scores, dialogue acts etc. A sequence of hypothe- ses forms a hypotheses sequence and finally several (alternate) hypotheses sequences form a hypothesis lattice. 2.1 Domain Model The processing of discourse information is not in- dependent of the representation formalism. Thus we will briefly characterize the approach taken in SmartKom, see (Gurevych et al., 2002) and also section 4. The representation is similar to frame- based approaches (Minsky, 1975). It is based on an ontology of application objects and subobjects. Application objects, e. g., a movie ticket reservation event, contain subobjects, e. g., movie theater, show time, and movie information. Subobjects are recursively composed of other subobjects, e. g., the movie theater subobject contains contact informa- tion which includes an address which includes a street name etc. The list of top-level objects, i. e., application objects is of course determined by the set of applications. In SmartKom, there are cur- rently about 10 different application objects. In ad-