Patterns of bibliographic references in the ACM published papers Jacques Wainer * , Henrique Przibisczki de Oliveira, Ricardo Anido Computing Institute, University of Campinas, Av. Albert Einstein 1251, 13083-852 Campinas, SP, Brazil article info Article history: Received 4 January 2010 Received in revised form 14 July 2010 Accepted 29 July 2010 Available online 25 August 2010 Keywords: Bibliometrics Citations Computer Science Conference papers abstract This paper analyzes the bibliographic references made by all papers published by ACM in 2006. Both an automatic classification of all references and a human classification of a ran- dom sample of them resulted that around 40% of the references are to conference proceed- ings papers, around 30% are to journal papers, and around 8% are to books. Among the other types of documents, standards and RFC correspond to 3% of the references, technical and other reports correspond to 4%, and other Web references to 3%. Among the documents cited at least 10 times by the 2006 ACM papers, 41% are conferences papers, 37% are books, and 16% are journal papers. Ó 2010 Elsevier Ltd. All rights reserved. 1. Citation analysis in Computer Science A scientific document, which we will call a source, makes references to other scientific documents (which we will call des- tinations) in order to acknowledge the relation of its content with the wider scientific field. In Computer Science (CS) tradi- tion, references are listed at the end of the paper, in a references or bibliography section. If a source document A makes a reference to a destination document B, we will say that B was cited (by A). Thus, in this paper we will use cited to refer to destinations. Scientific documents may be published in different venues – such as journals, conferences proceedings, tech- nical reports, books and book chapters, which we call document types. Citation analysis is the study of the references mainly from the destination’s point of view. The intuition is that important results are cited many times, and thus the count of how many times a paper, an author, or a journal have been cited is a proxy for its ‘‘importance.” Citation analysis has been used more commonly as a tool to evaluate journals (Garfield, 1972), universities, departments, and researchers (Moed, 2005; Ren and Taylor, 2007). On the other hand, CS research is not well represented by the standard research evaluation metrics; Parnas (2007) and Meyer et al. (2009), among others, have pointed out that CS research has a high respect for conference publications, which are not included in most citation count metrics and thus, places the field in disadvantage when compared with other Sciences. CS researchers agree that conferences are important; some CS subareas attribute more prestige to having papers accepted to particular conferences than to journals. Many arguments have been put forth for the importance of conferences as a pub- lication venue for CS, including the speed of publications of results and the dynamic character of CS research areas which may not fit well with the standard view of a journal with a well defined, fixed scope of interest. One of the goals of this research is to evaluate how important are conference papers as references of CS papers. Do CS papers, in general, make references to conference papers? Are references to conference papers the majority of the references in CS papers? Are conference papers among the most cited destinations of CS papers? The answers to these questions provide us with some evidence of the importance of conference papers in Computer Science research. 0306-4573/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.ipm.2010.07.002 * Corresponding author. Tel.: +55 19 3521 5871; fax: +55 19 3521 5847. E-mail addresses: wainer@ic.unicamp.br (J. Wainer), henrique.prz@gmail.com (H. Przibisczki de Oliveira), ranido@ic.unicamp.br (R. Anido). Information Processing and Management 47 (2011) 135–142 Contents lists available at ScienceDirect Information Processing and Management journal homepage: www.elsevier.com/locate/infoproman