Assassins and zealots: Variations in peer review: Special report

Stanley S. Siegelman

Research output: Contribution to journalReview article

Abstract

Radiology reviewers are required to assign numerical grades of 1 to 9 (1 = accept, 9 = reject) in the rating of manuscripts. The mean ratings for the 660 referees who were assigned 10 or more reviews over a 41/2-year period were analyzed. The mean score was 4.8 ± 0.8, and 87.4% of reviewers (the mainstream) had ratings of mean ± 1.5 standard deviations. Categories of reviewers with greater deviation from the mean were identified: zealots and pushovers, whose ratings of manuscripts were more favorable, versus assassins and demoters, who supplied less favorable ratings. To exclude the possibility that the referees who were classified as more critical had actually been sent less meritorious papers, the scores and rejection rates of 859 papers co-reviewed by assassins, demoters, and mainstream referees were compared. Significant differences were confirmed. Deviant referees were widely distributed in the pool of reviewers, including 13 members of the Editorial Board and representatives in each of 19 subspecialty areas. Failure to recognize and control for reviewer variation may be unfair to authors. An Editor has the capacity to reduce unfairness by monitoring reviewer variation and by modulating the review process accordingly.

Original languageEnglish (US)
Pages (from-to)637-642
Number of pages6
JournalRADIOLOGY
Volume178
Issue number3
DOIs
StatePublished - Mar 1991

Keywords

  • Radiology (journal)
  • Radiology and radiologists, research

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Fingerprint Dive into the research topics of 'Assassins and zealots: Variations in peer review: Special report'. Together they form a unique fingerprint.

  • Cite this