Learned Publishing, 25: 232–234 doi:10.1087/20120311 Wanted: a ‘Reviewer Effectiveness Index’ Philippe C. Baveye LEARNED PUBLISHING VOL. 25 NO. 3 JULY 2012 Another yearly individual perfor- mance assessment exercise has gone by in universities and research insti- tutes around the world, and, once again, in the overwhelming majority of cases, a crucial aspect of research- ers’ activity has been virtually ignored. As a rule, universities and research centers require data on grant monies awarded to their employees (or simply the amount of overhead they generated), on num- ber of articles, books, or book chapters published, on awards, and on the number of invited talks. Information about committee mem- berships, internally or in national/ international organizations, is com- monly requested. To the extent that they can bring prestige (and, there- fore, ultimately money) to their institution, administrators are also interested in any editorial responsi- bilities researchers have. However, few and far between are the institu- tions that solicit information of any kind about the anonymous, behind- the-scene work of peer reviewing manuscripts for journals, or propos- als for funding agencies. And yet, peer reviewing, a process that has often been criticized 1 but for which no viable alternative has ever been proposed, occupies a size- able portion of researchers’ time. A quick, back-of-the-envelope calcu- lation demonstrates this clearly. Most journals and funding agencies require between two and four peer reviews for each manuscript or proposal they have to evaluate. Therefore, if one takes multi-author- ship into account, as well as the fact that more manuscripts or proposals are submitted than are eventually published or funded, each resear- cher, on average, should probably review roughly three times as many manuscripts and research proposals as he/she is submitting. Someone publishing, say, five articles and sub- mitting two proposals per year should be willing to review at least 15 manuscripts and 6 proposals, i.e. carry out 21 reviews in total per year. The time spent on each of them depends on the type of manuscript or proposal involved, on the amount (if any) of background reading needed to produce a good review, and on the level of detail with which the peer-review report is put together. An average of about 6 hours per review, from start to fin- ish, seems reasonable, with some people spending considerably longer than that. Given that estimate, the individual mentioned above should theoretically devote roughly 126 hours per year to peer reviewing, equivalent to slightly over three full, 40-hours weeks of work, which is a sizeable fraction of his/her overall effort. 2 In the last few years, not without a sense of alarm at this development, journal editors in a wide range of disciplines have reported increasing difficulties in finding peer review- ers. 3 Statistics are lacking to deter- mine how serious the problem is. Likewise, it is not entirely clear what other causes of the problem there may be, besides the lack of recogni- tion of the peer-review process. Certainly, the fact that the pressure on researchers to attract grant mon- ies has increased dramatically over the last decade, and that the number of manuscripts submitted for publi- cation has skyrocketed (in particular from countries such as China or India that until recently did not have a strong publishing tradition) 4 must be contributing factors. Never- theless, regardless of the lack of information on the peer-review cri- sis, the perception of editors, ‘from the trenches’, seems difficult to dis- miss. Their day-to-day observation is that, unlike in years passed, it is becoming less and less exceptional for them to have to issue 10–15 invi- tations before they can secure the peer reviews needed to assess a given manuscript. Most often, researchers declining invitations to review invoke the fact that they are too busy to add yet another item to their already overcommitted schedule. In 232 Philippe C. Baveye LEARNED PUBLISHING VOL. 25 NO. 3 JULY 2012 POINT OF VIEW Wanted: a ‘Reviewer Effectiveness Index’ Philippe C. BAVEYE