Bias in neuroradiology peer review: Impact of a “ding” on “dinging” others

P. Charkhchi, B. Wang, Brian S Caffo, David Mark Yousem

Research output: Contribution to journalReview article

Abstract

BACKGROUND AND PURPOSE: The validity of radiology peer review requires an unbiased assessment of studies in an environment that values the process. We assessed radiologists' behavior reviewing colleagues' reports. We hypothesized that when a radiologist receives a discrepant peer review, he is more likely to submit a discrepant review about another radiologist. MATERIALS AND METHODS: We analyzed the anonymous peer review submissions of 13 neuroradiologists in semimonthly blocks of time from 2016 to 2018. We defined a discrepant review as any one of the following: 1) detection miss, clinically significant; 2) detection miss, clinically not significant; 3) interpretation miss, clinically significant; or 4) interpretation miss, clinically not significant. We used random-effects Poisson regression analysis to determine whether a neuroradiologist was more likely to submit a discrepant report during the semimonthly block in which he or she received one versus the semimonthly block thereafter. RESULTS: Four hundred sixty-eight discrepant peer review reports were submitted; 161 were submitted in the same semimonthly block of receipt of a discrepant report and 325 were not. Receiving a discrepant report had a positive effect on submitting discrepant reports: an expected relative increase of 14% (95% CI, 8%-21%). Notably, receiving a clinically not significant discrepant report (coefficient 0.13; 95% CI, 0.05- 0.22) significantly and positively correlated with submitting a discrepant report within the same time block, but this was not true of clinically significant reports. CONCLUSIONS: The receipt of a clinically not significant discrepant report leads to a greater likelihood of submitting a discrepant report. The motivation for such an increase should be explored for potential bias.

Original languageEnglish (US)
Pages (from-to)19-24
Number of pages6
JournalAmerican Journal of Neuroradiology
Volume40
Issue number1
DOIs
StatePublished - Jan 1 2019

Fingerprint

Peer Review
Radiology
Motivation
Regression Analysis
Radiologists

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Clinical Neurology

Cite this

Bias in neuroradiology peer review : Impact of a “ding” on “dinging” others. / Charkhchi, P.; Wang, B.; Caffo, Brian S; Yousem, David Mark.

In: American Journal of Neuroradiology, Vol. 40, No. 1, 01.01.2019, p. 19-24.

Research output: Contribution to journalReview article

@article{60a8bfed7125443f889f24d35fa4814a,
title = "Bias in neuroradiology peer review: Impact of a “ding” on “dinging” others",
abstract = "BACKGROUND AND PURPOSE: The validity of radiology peer review requires an unbiased assessment of studies in an environment that values the process. We assessed radiologists' behavior reviewing colleagues' reports. We hypothesized that when a radiologist receives a discrepant peer review, he is more likely to submit a discrepant review about another radiologist. MATERIALS AND METHODS: We analyzed the anonymous peer review submissions of 13 neuroradiologists in semimonthly blocks of time from 2016 to 2018. We defined a discrepant review as any one of the following: 1) detection miss, clinically significant; 2) detection miss, clinically not significant; 3) interpretation miss, clinically significant; or 4) interpretation miss, clinically not significant. We used random-effects Poisson regression analysis to determine whether a neuroradiologist was more likely to submit a discrepant report during the semimonthly block in which he or she received one versus the semimonthly block thereafter. RESULTS: Four hundred sixty-eight discrepant peer review reports were submitted; 161 were submitted in the same semimonthly block of receipt of a discrepant report and 325 were not. Receiving a discrepant report had a positive effect on submitting discrepant reports: an expected relative increase of 14{\%} (95{\%} CI, 8{\%}-21{\%}). Notably, receiving a clinically not significant discrepant report (coefficient 0.13; 95{\%} CI, 0.05- 0.22) significantly and positively correlated with submitting a discrepant report within the same time block, but this was not true of clinically significant reports. CONCLUSIONS: The receipt of a clinically not significant discrepant report leads to a greater likelihood of submitting a discrepant report. The motivation for such an increase should be explored for potential bias.",
author = "P. Charkhchi and B. Wang and Caffo, {Brian S} and Yousem, {David Mark}",
year = "2019",
month = "1",
day = "1",
doi = "10.3174/ajnr.A5908",
language = "English (US)",
volume = "40",
pages = "19--24",
journal = "American Journal of Neuroradiology",
issn = "0195-6108",
publisher = "American Society of Neuroradiology",
number = "1",

}

TY - JOUR

T1 - Bias in neuroradiology peer review

T2 - Impact of a “ding” on “dinging” others

AU - Charkhchi, P.

AU - Wang, B.

AU - Caffo, Brian S

AU - Yousem, David Mark

PY - 2019/1/1

Y1 - 2019/1/1

N2 - BACKGROUND AND PURPOSE: The validity of radiology peer review requires an unbiased assessment of studies in an environment that values the process. We assessed radiologists' behavior reviewing colleagues' reports. We hypothesized that when a radiologist receives a discrepant peer review, he is more likely to submit a discrepant review about another radiologist. MATERIALS AND METHODS: We analyzed the anonymous peer review submissions of 13 neuroradiologists in semimonthly blocks of time from 2016 to 2018. We defined a discrepant review as any one of the following: 1) detection miss, clinically significant; 2) detection miss, clinically not significant; 3) interpretation miss, clinically significant; or 4) interpretation miss, clinically not significant. We used random-effects Poisson regression analysis to determine whether a neuroradiologist was more likely to submit a discrepant report during the semimonthly block in which he or she received one versus the semimonthly block thereafter. RESULTS: Four hundred sixty-eight discrepant peer review reports were submitted; 161 were submitted in the same semimonthly block of receipt of a discrepant report and 325 were not. Receiving a discrepant report had a positive effect on submitting discrepant reports: an expected relative increase of 14% (95% CI, 8%-21%). Notably, receiving a clinically not significant discrepant report (coefficient 0.13; 95% CI, 0.05- 0.22) significantly and positively correlated with submitting a discrepant report within the same time block, but this was not true of clinically significant reports. CONCLUSIONS: The receipt of a clinically not significant discrepant report leads to a greater likelihood of submitting a discrepant report. The motivation for such an increase should be explored for potential bias.

AB - BACKGROUND AND PURPOSE: The validity of radiology peer review requires an unbiased assessment of studies in an environment that values the process. We assessed radiologists' behavior reviewing colleagues' reports. We hypothesized that when a radiologist receives a discrepant peer review, he is more likely to submit a discrepant review about another radiologist. MATERIALS AND METHODS: We analyzed the anonymous peer review submissions of 13 neuroradiologists in semimonthly blocks of time from 2016 to 2018. We defined a discrepant review as any one of the following: 1) detection miss, clinically significant; 2) detection miss, clinically not significant; 3) interpretation miss, clinically significant; or 4) interpretation miss, clinically not significant. We used random-effects Poisson regression analysis to determine whether a neuroradiologist was more likely to submit a discrepant report during the semimonthly block in which he or she received one versus the semimonthly block thereafter. RESULTS: Four hundred sixty-eight discrepant peer review reports were submitted; 161 were submitted in the same semimonthly block of receipt of a discrepant report and 325 were not. Receiving a discrepant report had a positive effect on submitting discrepant reports: an expected relative increase of 14% (95% CI, 8%-21%). Notably, receiving a clinically not significant discrepant report (coefficient 0.13; 95% CI, 0.05- 0.22) significantly and positively correlated with submitting a discrepant report within the same time block, but this was not true of clinically significant reports. CONCLUSIONS: The receipt of a clinically not significant discrepant report leads to a greater likelihood of submitting a discrepant report. The motivation for such an increase should be explored for potential bias.

UR - http://www.scopus.com/inward/record.url?scp=85059904139&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85059904139&partnerID=8YFLogxK

U2 - 10.3174/ajnr.A5908

DO - 10.3174/ajnr.A5908

M3 - Review article

C2 - 30523137

AN - SCOPUS:85059904139

VL - 40

SP - 19

EP - 24

JO - American Journal of Neuroradiology

JF - American Journal of Neuroradiology

SN - 0195-6108

IS - 1

ER -