Do readers and peer reviewers agree on manuscript quality?

Amy C. Justice, Jesse A. Berlin, Suzanne W. Fletcher, Robert H. Fletcher, Steven N. Goodman

Research output: Contribution to journalArticle

Abstract

Objective. - To study readers' judgments of manuscript quality and the degree to which readers agreed with peer reviewers. Design. - Cross- sectional study. Setting. - Annals of Internal Medicine. Subjects. - One hundred thirteen consecutive manuscripts reporting original research and selected for publication. Each of two manuscript versions (one before and one after revision) was judged by two readers, randomly sampled from those who said (based on the title) that they would read the article; one peer reviewer (peer), chosen in the usual way for Annals; and one expert in clinical research methods (expert). Each judge completed an instrument that included a 10-point subjective summary grade of manuscript quality. Main Outcome Measures. - Agreement on the 10-point summary grade of manuscript quality between reader-expert, reader-peer, and reader-reader. Results. - Readers and peers gave high grades (77% and 73% gave a grade of 5 or better, respectively), while experts were more critical (52% gave a grade of 5 or better; P69%) but agreement beyond chance was poor (κ

Original languageEnglish (US)
Pages (from-to)117-119
Number of pages3
JournalJournal of the American Medical Association
Volume272
Issue number2
DOIs
StatePublished - Jul 13 1994

ASJC Scopus subject areas

  • Medicine(all)

Fingerprint Dive into the research topics of 'Do readers and peer reviewers agree on manuscript quality?'. Together they form a unique fingerprint.

  • Cite this

    Justice, A. C., Berlin, J. A., Fletcher, S. W., Fletcher, R. H., & Goodman, S. N. (1994). Do readers and peer reviewers agree on manuscript quality? Journal of the American Medical Association, 272(2), 117-119. https://doi.org/10.1001/jama.272.2.117