Variability among observers utilizing the CellSolutions BestCyte Cell Sorter imaging system for the assessment of urinary tract cytology specimens

Elise Gelwan, M. Lisa Zhang, Derek B. Allison, Morgan L. Cowan, Juliana DeLuca, Joshua Fite, Sintawat Wangsiricharoen, Bonnie Williamson, Amy Zhou, Christopher VandenBussche

Research output: Contribution to journalArticle

Abstract

Introduction: Image analysis systems are not currently commonly used for evaluating urinary cytology specimens. We evaluated whether the BestCyte Cell Sorter (CellSolutions, Greensboro, NC) imaging system can reliably identify atypical cells in urinary cytology specimens. Methods: Fifty-three consecutive urine cytology specimens underwent 2 preparations: one slide using SurePath (SP; BD Diagnostics, Sparks, MD) for routine clinical evaluation, and a second slide using the CellSolutions F50 system for analysis by the BestCyte Cell Sorter (BCCS) scanning system. Eight observers reviewed atypical cells flagged by BCCS and assigned a BCCS diagnosis to each of the 53 specimens. The observers also blindly reviewed the SP preparation (when available) and assigned an SP diagnosis. The SP diagnoses given by one “expert” observer was considered as a reference diagnosis. Results: There was fair-to-moderate agreement among observers for identifying any atypia and high-grade atypia (Fleiss kappa: 0.417 and 0.338, respectively) using BCCS. Review of SP preparations had slightly better agreement (Fleiss kappa: 0.558 and 0.564, respectively). Intraobserver agreement between the two methods varied greatly between individuals (Cohen's kappa range: 0.260 to 0.647). When a consensus diagnosis could be reached among the observers for cases with surgical follow-up, the consensus diagnosis was concordant in 11 of 12 instances, with one instance being a one-step discrepancy. Conclusions: Specimen review by BCCS resulted in slightly greater interobserver variability than review of routine SP preparations. This may have been due to variations in observer experience and comfort with the use of a digital imaging system, which is further suggested by the wide range of intraobserver agreement among individuals.

Original languageEnglish (US)
JournalJournal of the American Society of Cytopathology
DOIs
StateAccepted/In press - Jan 1 2018

Fingerprint

Urinary Tract
Cell Biology
Observer Variation
Systems Analysis
Urine

Keywords

  • Bladder cancer
  • Digital imaging
  • Urine
  • Urothelial carcinoma
  • Urothelial neoplasia

ASJC Scopus subject areas

  • Pathology and Forensic Medicine

Cite this

Variability among observers utilizing the CellSolutions BestCyte Cell Sorter imaging system for the assessment of urinary tract cytology specimens. / Gelwan, Elise; Zhang, M. Lisa; Allison, Derek B.; Cowan, Morgan L.; DeLuca, Juliana; Fite, Joshua; Wangsiricharoen, Sintawat; Williamson, Bonnie; Zhou, Amy; VandenBussche, Christopher.

In: Journal of the American Society of Cytopathology, 01.01.2018.

Research output: Contribution to journalArticle

Gelwan, Elise ; Zhang, M. Lisa ; Allison, Derek B. ; Cowan, Morgan L. ; DeLuca, Juliana ; Fite, Joshua ; Wangsiricharoen, Sintawat ; Williamson, Bonnie ; Zhou, Amy ; VandenBussche, Christopher. / Variability among observers utilizing the CellSolutions BestCyte Cell Sorter imaging system for the assessment of urinary tract cytology specimens. In: Journal of the American Society of Cytopathology. 2018.
@article{4cc4fb9a4d384d0c96aff840d4f6fb9b,
title = "Variability among observers utilizing the CellSolutions BestCyte Cell Sorter imaging system for the assessment of urinary tract cytology specimens",
abstract = "Introduction: Image analysis systems are not currently commonly used for evaluating urinary cytology specimens. We evaluated whether the BestCyte Cell Sorter (CellSolutions, Greensboro, NC) imaging system can reliably identify atypical cells in urinary cytology specimens. Methods: Fifty-three consecutive urine cytology specimens underwent 2 preparations: one slide using SurePath (SP; BD Diagnostics, Sparks, MD)™ for routine clinical evaluation, and a second slide using the CellSolutions F50 system for analysis by the BestCyte Cell Sorter (BCCS) scanning system. Eight observers reviewed atypical cells flagged by BCCS and assigned a BCCS diagnosis to each of the 53 specimens. The observers also blindly reviewed the SP preparation (when available) and assigned an SP diagnosis. The SP diagnoses given by one “expert” observer was considered as a reference diagnosis. Results: There was fair-to-moderate agreement among observers for identifying any atypia and high-grade atypia (Fleiss kappa: 0.417 and 0.338, respectively) using BCCS. Review of SP preparations had slightly better agreement (Fleiss kappa: 0.558 and 0.564, respectively). Intraobserver agreement between the two methods varied greatly between individuals (Cohen's kappa range: 0.260 to 0.647). When a consensus diagnosis could be reached among the observers for cases with surgical follow-up, the consensus diagnosis was concordant in 11 of 12 instances, with one instance being a one-step discrepancy. Conclusions: Specimen review by BCCS resulted in slightly greater interobserver variability than review of routine SP preparations. This may have been due to variations in observer experience and comfort with the use of a digital imaging system, which is further suggested by the wide range of intraobserver agreement among individuals.",
keywords = "Bladder cancer, Digital imaging, Urine, Urothelial carcinoma, Urothelial neoplasia",
author = "Elise Gelwan and Zhang, {M. Lisa} and Allison, {Derek B.} and Cowan, {Morgan L.} and Juliana DeLuca and Joshua Fite and Sintawat Wangsiricharoen and Bonnie Williamson and Amy Zhou and Christopher VandenBussche",
year = "2018",
month = "1",
day = "1",
doi = "10.1016/j.jasc.2018.10.001",
language = "English (US)",
journal = "Journal of the American Society of Cytopathology",
issn = "2213-2945",
publisher = "Elsevier BV",

}

TY - JOUR

T1 - Variability among observers utilizing the CellSolutions BestCyte Cell Sorter imaging system for the assessment of urinary tract cytology specimens

AU - Gelwan, Elise

AU - Zhang, M. Lisa

AU - Allison, Derek B.

AU - Cowan, Morgan L.

AU - DeLuca, Juliana

AU - Fite, Joshua

AU - Wangsiricharoen, Sintawat

AU - Williamson, Bonnie

AU - Zhou, Amy

AU - VandenBussche, Christopher

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Introduction: Image analysis systems are not currently commonly used for evaluating urinary cytology specimens. We evaluated whether the BestCyte Cell Sorter (CellSolutions, Greensboro, NC) imaging system can reliably identify atypical cells in urinary cytology specimens. Methods: Fifty-three consecutive urine cytology specimens underwent 2 preparations: one slide using SurePath (SP; BD Diagnostics, Sparks, MD)™ for routine clinical evaluation, and a second slide using the CellSolutions F50 system for analysis by the BestCyte Cell Sorter (BCCS) scanning system. Eight observers reviewed atypical cells flagged by BCCS and assigned a BCCS diagnosis to each of the 53 specimens. The observers also blindly reviewed the SP preparation (when available) and assigned an SP diagnosis. The SP diagnoses given by one “expert” observer was considered as a reference diagnosis. Results: There was fair-to-moderate agreement among observers for identifying any atypia and high-grade atypia (Fleiss kappa: 0.417 and 0.338, respectively) using BCCS. Review of SP preparations had slightly better agreement (Fleiss kappa: 0.558 and 0.564, respectively). Intraobserver agreement between the two methods varied greatly between individuals (Cohen's kappa range: 0.260 to 0.647). When a consensus diagnosis could be reached among the observers for cases with surgical follow-up, the consensus diagnosis was concordant in 11 of 12 instances, with one instance being a one-step discrepancy. Conclusions: Specimen review by BCCS resulted in slightly greater interobserver variability than review of routine SP preparations. This may have been due to variations in observer experience and comfort with the use of a digital imaging system, which is further suggested by the wide range of intraobserver agreement among individuals.

AB - Introduction: Image analysis systems are not currently commonly used for evaluating urinary cytology specimens. We evaluated whether the BestCyte Cell Sorter (CellSolutions, Greensboro, NC) imaging system can reliably identify atypical cells in urinary cytology specimens. Methods: Fifty-three consecutive urine cytology specimens underwent 2 preparations: one slide using SurePath (SP; BD Diagnostics, Sparks, MD)™ for routine clinical evaluation, and a second slide using the CellSolutions F50 system for analysis by the BestCyte Cell Sorter (BCCS) scanning system. Eight observers reviewed atypical cells flagged by BCCS and assigned a BCCS diagnosis to each of the 53 specimens. The observers also blindly reviewed the SP preparation (when available) and assigned an SP diagnosis. The SP diagnoses given by one “expert” observer was considered as a reference diagnosis. Results: There was fair-to-moderate agreement among observers for identifying any atypia and high-grade atypia (Fleiss kappa: 0.417 and 0.338, respectively) using BCCS. Review of SP preparations had slightly better agreement (Fleiss kappa: 0.558 and 0.564, respectively). Intraobserver agreement between the two methods varied greatly between individuals (Cohen's kappa range: 0.260 to 0.647). When a consensus diagnosis could be reached among the observers for cases with surgical follow-up, the consensus diagnosis was concordant in 11 of 12 instances, with one instance being a one-step discrepancy. Conclusions: Specimen review by BCCS resulted in slightly greater interobserver variability than review of routine SP preparations. This may have been due to variations in observer experience and comfort with the use of a digital imaging system, which is further suggested by the wide range of intraobserver agreement among individuals.

KW - Bladder cancer

KW - Digital imaging

KW - Urine

KW - Urothelial carcinoma

KW - Urothelial neoplasia

UR - http://www.scopus.com/inward/record.url?scp=85055970001&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055970001&partnerID=8YFLogxK

U2 - 10.1016/j.jasc.2018.10.001

DO - 10.1016/j.jasc.2018.10.001

M3 - Article

C2 - 30929755

AN - SCOPUS:85055970001

JO - Journal of the American Society of Cytopathology

JF - Journal of the American Society of Cytopathology

SN - 2213-2945

ER -