Measuring research excellence Number of Nobel Prize achievements versus conventional bibliometric indicators Alonso Rodrı ´guez-Navarro Centro de Biotecnologı ´a y Geno ´mica de Plantas, Universidad Polite ´cnica de Madrid, Campus de Montegancedo, Spain Abstract Purpose – Several bibliometric indicators that are extensively used to estimate research performance have not been validated against an external criterion of excellence. This paper aims to investigate whether this validation is possible using the number of Nobel Prize awards. Design/methodology/approach – This study uses several analytical treatments of the data to investigate: whether Nobel Prize awards are sporadic events or they depend on the scientific activity of countries or institutions and can be used in research evaluation; and the association between the number of Nobel Prize achievements and conventional bibliometric indicators across countries and institutions. Findings – This study finds that conventional bibliometric indicators, numbers of publications, citations, and top 1 per cent most cited publications, correlate with the number of Nobel Prize achievements in several advanced countries with similar research abilities. Contrarily, in countries and institutions with more variable research characteristics, there is no association between conventional bibliometric indicators and the number of Nobel Prize achievements, and their use as indicators of research excellence is not valid. In contrast, the number of national articles in Nature and Science correlates with the number of Nobel Prize achievements across countries and institutions. Practical implications – Science administrators implementing research evaluations and research incentives based on conventional bibliometric indicators should consider that increasing the scores of these indicators does not imply an improvement in research excellence. Originality/value – The study demonstrates that Nobel Prize achievements are not singular events that occur by chance. Therefore, the number of Nobel Prize achievements can be used to validate bibliometric indicators. Keywords Bibliometric indicators, Research performance, Nobel prizes, Research results, Research Paper type Research paper 1. Introduction Research sustains innovation and innovation is one of the main driving forces behind economic growth. Therefore, the ability to estimate research performance is vital for governments to know the real benefits of their research investments. Two substantially different methods are normally used for this estimation: bibliometric analysis of research output, such as number and share of publications and citations (see Almeida et al., 2009, and references therein), and metric of internationally recognized scientific prizes, especially Nobel Prizes (Charlton, 2007a, b, c). Using Thomas Kuhn’s categories of science (Kuhn, 1970) in a broad sense, the number of The current issue and full text archive of this journal is available at www.emeraldinsight.com/0022-0418.htm The author thanks Ignacio Romagosa, Fernando Garcı ´a-Arenal, and Juan Orellana for their helpful comments and suggestions. JDOC 67,4 582 Received 17 June 2010 Revised 27 August 2010 Accepted 27 August 2010 Journal of Documentation Vol. 67 No. 4, 2011 pp. 582-600 q Emerald Group Publishing Limited 0022-0418 DOI 10.1108/00220411111145007