TY - JOUR
T1 - Accuracy and Interrater Reliability of Point-of-Care Ultrasonography Image Interpretation for Intussusception
AU - In Collaboration with the P2Network Intussusception Study Group
AU - Bergmann, Kelly R.
AU - Khant, Marshal
AU - Lammers, Shea
AU - Arroyo, Alexander C.
AU - Avendano, Pablo
AU - Chaudoin, Lindsey
AU - Cohen, Stephanie G.
AU - Deanehan, J. Kate
AU - Kornblith, Aaron E.
AU - Lam, Samuel H.F.
AU - Lin-Martore, Margaret
AU - Malia, Laurie
AU - Pade, Kathryn H.
AU - Park, Daniel B.
AU - Sivitz, Adam
AU - Shahar-Nissan, Keren
AU - Snelling, Peter J.
AU - Tessaro, Mark O.
AU - Thomas-Mohtat, Rosemary
AU - Whitcomb, Valerie
AU - Yock-Corrales, Adriana
AU - Walsh, Paige
AU - Watson, Dave
AU - Madhok, Manu
N1 - Funding Information:
Funding source: This work was supported (in part) by a grant from the Research Committee of Children's Hospitals and Clinics of Minnesota.
Publisher Copyright:
© Wolters Kluwer Health, Inc. All rights reserved.
PY - 2022/9/1
Y1 - 2022/9/1
N2 - Objectives The aim of this study was to determine the accuracy and interrater reliability of (1) point-of-care ultrasound (POCUS) image interpretation for identification of intussusception and (2) reliability of secondary signs associated with intussusception among experts compared with novice POCUS reviewers. Methods We conducted a planned secondary analysis of a prospective, convenience sample of children aged 3 months to 6 years who were evaluated with POCUS for intussusception across 17 international pediatric emergency departments between October 2018 and December 2020. A random sample of 100 POCUS examinations was reviewed by novice and expert POCUS reviewers. The primary outcome was identification of the presence or absence of intussusception. Secondary outcomes included intussusception size and the presence of trapped free fluid or echogenic foci. Accuracy was summarized using sensitivity and specificity, which were estimated via generalized mixed effects logistic regression. Interrater reliability was summarized via Light's κ statistics with bootstrapped standard errors (SEs). Accuracy and reliability of expert and novice POCUS reviewers were compared. Results Eighteen expert and 16 novice POCUS reviewers completed the reviews. The average expert sensitivity was 94.5% (95% confidence interval [CI], 88.6-97.5), and the specificity was 94.3% (95% CI, 90.3-96.7), significantly higher than the average novice sensitivity of 84.7% (95% CI, 74.3-91.4) and specificity of 80.4% (95% CI, 72.4, 86.7). κ was significantly greater for expert (0.679, SE 0.039) compared with novice POCUS reviewers (0.424, SE 0.044; difference 0.256, SE 0.033). For our secondary outcome measure of intussusception size, κ was significantly greater for experts (0.661, SE 0.038) compared with novices (0.397, SE 0.041; difference 0.264, SE 0.029). Interrater reliability was weak for expert and minimal for novice reviewers regarding the detection of trapped free fluid and echogenic foci. Conclusions Expert POCUS reviewers demonstrate high accuracy and moderate interrater reliability when identifying intussusception via image interpretation and perform better than novice reviewers.
AB - Objectives The aim of this study was to determine the accuracy and interrater reliability of (1) point-of-care ultrasound (POCUS) image interpretation for identification of intussusception and (2) reliability of secondary signs associated with intussusception among experts compared with novice POCUS reviewers. Methods We conducted a planned secondary analysis of a prospective, convenience sample of children aged 3 months to 6 years who were evaluated with POCUS for intussusception across 17 international pediatric emergency departments between October 2018 and December 2020. A random sample of 100 POCUS examinations was reviewed by novice and expert POCUS reviewers. The primary outcome was identification of the presence or absence of intussusception. Secondary outcomes included intussusception size and the presence of trapped free fluid or echogenic foci. Accuracy was summarized using sensitivity and specificity, which were estimated via generalized mixed effects logistic regression. Interrater reliability was summarized via Light's κ statistics with bootstrapped standard errors (SEs). Accuracy and reliability of expert and novice POCUS reviewers were compared. Results Eighteen expert and 16 novice POCUS reviewers completed the reviews. The average expert sensitivity was 94.5% (95% confidence interval [CI], 88.6-97.5), and the specificity was 94.3% (95% CI, 90.3-96.7), significantly higher than the average novice sensitivity of 84.7% (95% CI, 74.3-91.4) and specificity of 80.4% (95% CI, 72.4, 86.7). κ was significantly greater for expert (0.679, SE 0.039) compared with novice POCUS reviewers (0.424, SE 0.044; difference 0.256, SE 0.033). For our secondary outcome measure of intussusception size, κ was significantly greater for experts (0.661, SE 0.038) compared with novices (0.397, SE 0.041; difference 0.264, SE 0.029). Interrater reliability was weak for expert and minimal for novice reviewers regarding the detection of trapped free fluid and echogenic foci. Conclusions Expert POCUS reviewers demonstrate high accuracy and moderate interrater reliability when identifying intussusception via image interpretation and perform better than novice reviewers.
KW - intussusception
KW - point-of-care ultrasound
UR - http://www.scopus.com/inward/record.url?scp=85137126971&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85137126971&partnerID=8YFLogxK
U2 - 10.1097/PEC.0000000000002786
DO - 10.1097/PEC.0000000000002786
M3 - Review article
C2 - 36040465
AN - SCOPUS:85137126971
SN - 0749-5161
VL - 38
SP - 442
EP - 447
JO - Pediatric emergency care
JF - Pediatric emergency care
IS - 9
ER -