Evaluation of User Interaction in Digital Libraries Giannis Tsakonas, Sarantos Kapidakis, Christos Papatheodorou Laboratory of Digital Libraries and Electronic Publishing, Dept. of Archive and Library Sciences, Ionian University Palea Anaktora, Eleftherias Sq., Corfu, 491 00, Greece. {gtsak, sarantos, papatheodor} @ionio.gr Abstract. Digital library interaction evaluation has, by nature, a user centered perspective, which directs the scientific research to the benefit of the user and narrows the research parameters. In this paper we propose a new framework for the evaluation of interaction and we introduce the concepts of usefulness, usability and performance as the main contributing factors for the creation of a rational and successful interaction. The framework attempts to trace and highlight some of the interaction evaluation requirements and parameters, in order to be sufficient in many different contexts. 1. Introduction Digital libraries have been introduced to the daily life of a significant part of the scientific, educational and research population. These systems have left the narrow walls of the laboratory environment and address the real needs of users. The usage monitoring of these systems provide useful information for their evaluation and improvement processes. However there is a plethora of inter-dependent factors influencing the overall user interaction with a digital library and therefore there is a vital need for a holistic approach in evaluation, which should consider every aspect (agent, attribute, level etc) that may affect the interaction process. Digital library interaction shares experiences from two distinct communities, the Human Computer Interaction (HCI) and the information science community. The HCI community is carrying the expertise on the improvement of user interaction with a new information management medium, while the information science community adds the scent of domain knowledge in the sense of information behaviour. This conjunction imposes the investigation of the iterative exchange of dialogue elements between the user and a digital library system, which are translated through an interface and aims to fulfill the user informational needs. Digital library evaluation has many facets depending on the characteristics and the perspective of the evaluating agent. Borgman states that “the methods and metrics for evaluating digital libraries will vary by whether they are viewed as institutions, as information systems, as new technologies, or as new services” [1]. Moreover each developer, evaluator or project partner applies his own expertise over the evaluation domain and concentrate on the most important issues according to this expertise [2].