Mobile Delivery of News Using Hierarchical Query-Biased Summaries Simon O. Sweeney Department of Computer and Information Science University of Strathclyde Glasgow G1 1XH, UK simon@cis.strath.ac.uk Fabio Crestani Department of Computer and Information Sciences University of Strathclyde Glasgow G1 1XH, UK fabioc@cis.strath.ac.uk Anastasios Tombros Department of Computing Science University of Glasgow Glasgow G12 8QQ, UK tombrosa@dcs.gla.ac.uk ABSTRACT This paper presents the results of a study aimed at measur- ing the usefulness of presenting the results of an Information Retrieval search on WAP mobile phones. The experimen- tation focuses on presenting automatically-generated sum- maries of newspaper articles of increasing length on a mobile phone interface and in studying the differences in users’ per- ception of relevance of the retrieved documents. The aim is to study experimentally how users’ perception of relevance varies depending on the length of the summary, and in rela- tion to the specific characteristic of the mobile interface that content is presented. Experimental results suggest that hi- erarchical query biased summaries are useful when dealing with small screens and assist users in making correct rele- vance judgments. Keywords: WAP, document summarisation, user study. 1. INTRODUCTION AND MOTIVATIONS The development of the Internet as a global communica- tions medium has enabled millions of people to share infor- mation and conduct business electronically. In more recent years there has been increased demand for mobility - people want access to information resources and services at their demand, irrespective of location. Recent studies indicate that there are more than 300 million mobile phone users world-wide and current estimates predict significant growth for the future. The growing demand for mobility has led to the emergence of WAP (Wireless Application Protocol). WAP is seen as the next step in providing value added services to the cur- rent generation of mobile wireless devices. The types of ser- vices that WAP is intended to provide include such things as news and information resources, shopping, banking services, weather and traffic alerts, mapping, locator services and di- rectory services. A useful service for a hand held wireless Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC 2002, Madrid, Spain Copyright 2002 ACM 1-58113-445-2/02/03 ...$5.00. device would be the possibility of retrieving useful informa- tion pertinent to a query, allowing a user the flexibility to satisfy personal information requirements. The subject of Information Retrieval (IR) is well estab- lished and is an underlying feature of the Internet. The function of an IR system is to locate and retrieve relevant data that is subsequently presented to the user. The out- put of the IR system is assessed by the user on the basis of relevance to the query submitted. In normal circumstances an IR system presents its results on a computer, where the interface for the user is a monitor with a large screen dis- play, nominally 14-19 inches and interaction is by a mouse or QWERTY keyboard. In comparison, a WAP mobile phone interface is very d- ifferent, having a small display screen that is less powerful, and input/navigation controlled by the phone keypad [10, 4]. Because of these differences, the ability of a user to make relevance assessments of content viewed on a WAP mobile phone may be affected by these characteristics of the dis- play and of the interaction. It is the intention of this study to partially assess the effect of this form-factor by assessing the usefulness of viewing textual content on a mobile phone interface. In addition, due to the limitation in size of the screen of WAP phones (see further on for typical sizes and formats), there is need to reduce the amount of informa- tion presented to the user. This reduction in information presented, however, should not reduce the informativeness of the data presented. Therefore, a further motivation of the study is the assessment of the automatic summarisation of documents. The demand for the summarisation of docu- ments reflects the need to reduce the time required to extract useful information from the full text of a document. Consid- eration for the design and implementation of automatic sum- marisation systems is well documented and much research has been conducted on assessing their effectiveness [15, 14]. It is the intention of this study to evaluate the effectiveness of summaries of variable length. This paper is structured as follows. Section2 describes the background and presents some of the issues related to query- biased summarisation. Section 3 reports how the hierarchi- cal summaries were produced. Section 4 then presents the details of our experimental set-up. Subsequently, Section 5 presents the experimental results and analysis. Finally, Sec- tion 6 reports the conclusions from our work and presents future extensions of this work.