Design and evaluation of awareness mechanisms in CiteSeer Umer Farooq * , Craig H. Ganoe, John M. Carroll, Isaac G. Councill, C. Lee Giles College of Information Sciences and Technology, The Pennsylvania State University, University Park, PA 16802, USA Received 25 October 2006; received in revised form 8 May 2007; accepted 18 May 2007 Available online 16 July 2007 Abstract Awareness has been extensively studied in human computer interaction (HCI) and computer supported cooperative work (CSCW). The success of many collaborative systems hinges on effectively supporting awareness of different collab- orators, their actions, and the process of creating shared work products. As digital libraries are increasingly becoming more than just repositories for information search and retrieval – essentially fostering collaboration among its community of users – awareness remains an unexplored research area in this domain. We are investigating awareness mechanisms in CiteSeer, a scholarly digital library for the computer and information science domain. CiteSeer users can be notified of new publication events (e.g., publication of a paper that cites one of their papers) using feeds as notification systems. We pres- ent three cumulative user studies – requirements elicitation, prototype evaluation, and naturalistic study – in the context of supporting CiteSeer feeds. Our results indicate that users prefer feeds that place target items in query-relevant contexts, and that preferred context varies with type of publication event. We found that users integrated feeds as part of their broader, everyday activities and used them as planning tools to collaborate with others. Ó 2007 Elsevier Ltd. All rights reserved. Keywords: Computer-supported awareness; Notification systems; Communities of practice; Scholarly digital libraries 1. Introduction Scholars have always organized themselves into communities, within which they collaborate, debate, and otherwise interact. Today, the number and variety of communities, forums, and channels for such interactions are vast. Indeed, the recent growth of interdisciplinary scholarship – particularly in the sciences – has made contemporary scholarly communities more open, but also less identifiable. The network of people and resources relevant to one’s current scholarly project is quite dynamic; through the course of a career, or even of a project, one may move through a series of foci. Thus, it is now typical for scholars to wonder what com- munities are addressing issues of interest to them, and even what communities they are influencing. A concrete way to think about this challenge is that contemporary scholars need to be aware of a far wider range of colleagues and research topics. But clearly the solution is not to thoroughly read or even manually browse ever-widening swaths of research. That is not humanly possible. Fortunately, these new needs are 0306-4573/$ - see front matter Ó 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.ipm.2007.05.009 * Corresponding author. E-mail address: ufarooq@ist.psu.edu (U. Farooq). Available online at www.sciencedirect.com Information Processing and Management 44 (2008) 596–612 www.elsevier.com/locate/infoproman