Paris Interobserver Reproducibility Study (PIRST)

Daniel F.I. Kurtycz; Güliz A. Barkan; Derek M. Pavelec; Dorothy L. Rosenthal; Eva M. Wojcik; Christopher J. VandenBussche; Kala Mangiulli; Matthew T. Olson

doi:10.1016/j.jasc.2018.02.005

Paris Interobserver Reproducibility Study (PIRST)

Daniel F.I. Kurtycz, Güliz A. Barkan, Derek M. Pavelec, Dorothy L. Rosenthal, Eva M. Wojcik, Christopher J. VandenBussche, Kala Mangiulli, Matthew T. Olson

School of Medicine

Research output: Contribution to journal › Article › peer-review

Abstract

Objectives: In concert with the 2015 publication of The Paris System for Urinary Cytopathology (TPS), a Web-based interobserver study, co-sponsored by the American Society of Cytopathology (ASC) and International Academy of Cytology (IAC), was performed to determine diagnostic agreement among volunteer participants and with the TPS author consensus. Material and Methods: Participants at various levels of training and certification were recruited through national and international cytopathology professional societies. Although the survey was open to all comers, potential participants were screened by two basic cytopathology questions. Information was collected on the level of training, practice patterns, and experience. Study participants evaluated 85 images (previously unpublished) chosen from the TPS atlas. These images spanned all diagnostic categories. Results: Of the 1993 attempts to access the survey, 1313 participants correctly answered the qualifying questions and were included in the survey. Respondents were concentrated in the United States, although many participants came from other countries. The majority of respondents were board-certified in anatomic pathology with cytopathology certification. A smaller number were cytotechnologists. Board-certified cytopathologists and specialist cytotechnologists outperformed other certifications. Practice type (academics versus non-academic), and country (US versus international) were not major factors in concordance. Diagnostic categories with the best agreement were Negative for High-Grade Urothelial Carcinoma (NHGUC; 71%), Low-Grade Urothelial Neoplasm (LGUN; 62%), and High-Grade Urothelial Carcinoma (HGUC; 57%). Indeterminate categories showed low concordance. Conclusions: The NHGUC, LGUN, and HGUC were most correlated with diagnostic agreement among observers. This study can serve as a baseline for future comparisons.

Original language	English (US)
Pages (from-to)	174-184
Number of pages	11
Journal	Journal of the American Society of Cytopathology
Volume	7
Issue number	4
DOIs	https://doi.org/10.1016/j.jasc.2018.02.005
State	Published - Jul 1 2018

Keywords

Bladder cancer
Interobserver reproducibility
Standardized Terminology System
The Paris System
Urine cytology
Web Based Image Study

ASJC Scopus subject areas

Pathology and Forensic Medicine

Access to Document

10.1016/j.jasc.2018.02.005

Cite this

Paris Interobserver Reproducibility Study (PIRST). / Kurtycz, Daniel F.I.; Barkan, Güliz A.; Pavelec, Derek M.; Rosenthal, Dorothy L.; Wojcik, Eva M.; VandenBussche, Christopher J.; Mangiulli, Kala; Olson, Matthew T.

In: Journal of the American Society of Cytopathology, Vol. 7, No. 4, 01.07.2018, p. 174-184.

Research output: Contribution to journal › Article › peer-review

@article{fc6c2bcfe5004c07a34342e3a0b4b9b7,

title = "Paris Interobserver Reproducibility Study (PIRST)",

abstract = "Objectives: In concert with the 2015 publication of The Paris System for Urinary Cytopathology (TPS), a Web-based interobserver study, co-sponsored by the American Society of Cytopathology (ASC) and International Academy of Cytology (IAC), was performed to determine diagnostic agreement among volunteer participants and with the TPS author consensus. Material and Methods: Participants at various levels of training and certification were recruited through national and international cytopathology professional societies. Although the survey was open to all comers, potential participants were screened by two basic cytopathology questions. Information was collected on the level of training, practice patterns, and experience. Study participants evaluated 85 images (previously unpublished) chosen from the TPS atlas. These images spanned all diagnostic categories. Results: Of the 1993 attempts to access the survey, 1313 participants correctly answered the qualifying questions and were included in the survey. Respondents were concentrated in the United States, although many participants came from other countries. The majority of respondents were board-certified in anatomic pathology with cytopathology certification. A smaller number were cytotechnologists. Board-certified cytopathologists and specialist cytotechnologists outperformed other certifications. Practice type (academics versus non-academic), and country (US versus international) were not major factors in concordance. Diagnostic categories with the best agreement were Negative for High-Grade Urothelial Carcinoma (NHGUC; 71%), Low-Grade Urothelial Neoplasm (LGUN; 62%), and High-Grade Urothelial Carcinoma (HGUC; 57%). Indeterminate categories showed low concordance. Conclusions: The NHGUC, LGUN, and HGUC were most correlated with diagnostic agreement among observers. This study can serve as a baseline for future comparisons.",

keywords = "Bladder cancer, Interobserver reproducibility, Standardized Terminology System, The Paris System, Urine cytology, Web Based Image Study",

author = "Kurtycz, {Daniel F.I.} and Barkan, {G{\"u}liz A.} and Pavelec, {Derek M.} and Rosenthal, {Dorothy L.} and Wojcik, {Eva M.} and VandenBussche, {Christopher J.} and Kala Mangiulli and Olson, {Matthew T.}",

note = "Funding Information: The major findings of this study were: (1) the single most significant feature affecting the performance of survey responders in the application of TPS was the diagnostic category of the image; (2) the highest levels of interobserver reproducibility and accuracy were seen for NHGUC and HGUC; (3) the lowest performance levels were seen in detecting non-urothelial, non-squamous neoplasms and the indeterminate categories AUC and SHGUC; (4) subspecialization in cytopathology showed an increase in diagnostic accuracy among both cytopathologists and cytotechnologists in univariate analysis but not in a multivariate analysis; (5) there was no discernable difference between the performance of cytotechnologists and cytopathologists at classifying images according to TPS; and (6) there was no association between practice type or experience level in the performance of respondents at classifying images according to TPS. A challenge of 85 images to the cytologic community demonstrated that the feature most clearly associated with the performance of morphology is the diagnostic category, with NHGUC and HGUC being the most likely categories to be categorized correctly. This finding is both expected and welcome because it reinforces both practical experience and the universal consensus on the distinct cytomorphology of HGUC as opposed to Low-Grade Urothelial Neoplasia, which has been shown to have poor interobserver variability. 12 The classification of reactive conditions, formerly considered “atypical”, as NHGUC concurrently increased the size of this category and likely increased interobserver reproducibility. This claim is difficult to quantify because there have not been many studies performed prior to TPS on this topic. 13 The extremely poor performance of TPS with recognizing the cytological features of tumors that are neither urothelial nor squamous reflects the morphologic subtlety of these tumors in urinary cytology. No classification system can substitute for a broad differential diagnosis and knowledge of the past medical history leading to an index of suspicion for a non-urothelial malignancy, including metastatic/extravesical lesions. The poor reproducibility of indeterminate categories in cytology classification systems is well documented and TPS is no exception. 8,12,14-16 Thus, the finding that AUC and SHGUC had worse interobserver reproducibility than NHGUC and HGUC was expected. More interesting is the observation that a substantial amount of ambiguity lies around SHGUC, with this category splitting evenly between HGUC and AUC. Although this may not appear desirable prima facie, it is indeed welcome if it represents a shift away from the AUC versus NHGUC boundary. If this is the case, then it is early evidence that emphasis on HGUC and the de-emphasis of Low-Grade Urothelial Neoplasm have moved the point of disagreement among observers to a more clinically actionable space. Reiterating, it is the intent of TPS to separate urothelial cytology into two groups, one negative for high-grade urothelial carcinoma and one worrisome or positive for high-grade urothelial carcinoma. This means that the AUC and SHGUC categories are confined to those samples worrisome for HGUC. In many laboratories it has been common practice to use atypia for an array of inflammatory, infectious, and reactive conditions as well as those cases indeterminate for neoplasia. In TPS this is no longer the case. Excluding all of the former cases and restricting the indeterminate for neoplasia (AUC) category to those with cells having an N:C ratio of 0.5 or higher should lead to decrease in the use of AUC. SHGUC is also relatively restricted and defined as having 5 to 10 non-degenerate, non-superficial cells with N:C ratios of 0.5 to 0.7 and coarse chromatin or irregular membranes. If NHGUC and AUC become clearly separated from one another, the relative risk of AUC for representing carcinoma becomes greater. Thus, the positive predictive value of AUC improves. Because the suspicious category of SHGUC will also not be contaminated by abnormalities less than worrisome for HGUC, its relative risk and positive predictive value also improves. The separation should lead to a greater diagnostic yield of HGUC from cystoscopies following AUC and SHGUC cytology. The TPS authors do believe that, because of decreased interpretations of AUC in cases of GU cytology, there will be fewer patients needing cystoscopy. Alternatively, the reset in risk and predictive value for AUC and SHGUC should lead to improved detection of HGUC for cystoscopies performed on patients with AUC and SHGUC samples. Most respondents replied that that their laboratories receive voided urine (98%), catheterized urine (81%), bladder wash specimens (85%), ureteral washing/brushing specimens (71%), and renal pelvis washing/brushing specimens (68%). This is in line with a previous a College of American Pathologists (CAP) survey where voided urine, bladder washing/barbotage, and catheterized urine specimens were found to represent 99.6%, 86.6%, and 88.5% of the urine samples, respectively. 17 In addition, 745 participants noted that specimens in their laboratories are processed using ThinPrep (64%), cytospin (42%), SurePath (17%), conventional smears (21%), and other preparatory techniques (6%); this is similar to the findings of the CAP survey where ThinPrep (57.4%) and cytospin (45.5%) were found to be the most commonly used preparation types in urinary cytology. 17 Most of the morphological evidence of TPS was based on studies performed on ThinPrep and SurePath samples; however, both of the aforementioned surveys highlight that the ThinPrep and cytospin are the most common preparation types. This will necessitate additional morphological studies to be performed on cytospin preparations prior to an updated publication of TPS. Another problematic area for TPS is adequacy. There is no general agreement in the literature and there is debate on number of cells or volume or a combination of volume and cellularity. Morphologist characteristics such as training, cytology board certification, practice setting, and exposure to urinary cytology samples were all examined in this study to determine the appeal and feasibility of TPS utilization to the community at large. Of these features, only cytopathology subspecialization as documented by cytopathology board certification appeared to portend significantly better interobserver reproducibility in univariate analysis. As expected, there is an intuitive rationale for better performance among board-certified cytopathologists because those individuals are more likely to have practiced by TPS norms—especially pertaining to NHGUC and HGUC—before TPS was officially released. Neither setting nor annual volume of urinary cytology samples appeared to correlate significantly with performance on classifying images according to TPS in any of the subanalyses. The generally accepted view that long experience or high volume improves accuracy is not consistent with the current data; nonetheless, it should be noted that at the time of this study TPS was new and that the study may have been the first exposure of many of the participants to TPS. Therefore, volume and experience may not have helped with concordance. Recent literature also indicates that many cytologists tend to overestimate N:C ratio. 18 On the other hand, a similar lack of concordance with volume and experience was seen in the GYN Bethesda Interobserver reproducibility study. 19 As to why it was the case with Bethesda GYN, there is only speculation. Contrary to TPS, there are decades of experience with the Bethesda GYN categories. In these reproducibility studies, and much of teaching, still images are used. These are not whole slide samples. The still images are not representative of actual clinical specimens in that locator skills are not challenged and are often seen at higher magnification than in clinical practice. There are additional acknowledged limitations to this study. This study was geographically concentrated. The majority of respondents to this online survey were from North America (68%); this is probably a result of the methods used in requesting participation. Although requests went through the Web site of the International Academy of Cytopathology and to societies based in Asia, most of the activity was carried out through the Web site and listserv of the American Society of Cytopathology. Additionally, the survey was in English. Thus, the survey is over-weighted with the North American–English speaking experience. Neither TPS, nor this study, included images relative to radiation change or chemotherapy. Because such situations could prove diagnostically challenging, future editions of TPS and interobserver reproducibility studies need to take these sorts of samples into account. The Paris System Consortium has a number of objectives for refinement of the system and a number of questions to be answered. These are outlined in the afterword of the TPS and provide direction for future research efforts. 7 Goals include defining relative risk of the diagnostic categories; restricting use of atypical (“inconclusive”) categories; examining what actually constitutes an adequate sample for urinary cytology; and assessing how well users apply the criteria of TPS. 20-23 This study helps establish a baseline in support of the latter objective. Of note, the reclassification of images was not one of the aims of this study. The authors intended to formally identify the less-reproducible categories and set the groundwork for investigations that will hopefully lead to improved clarity. TPS aims to progressively refine the definitions for the sake of better diagnosis. Changes in the images used for future editions of TPS will depend on studies investigating utility of the categories, the risk of malignancy for a given category, and evolving photography. An important aspect of this project was that this survey was performed solely by volunteer effort with help from the American Society of Cytopathology Central Office and the support of the Division of Information Technology of the University of Wisconsin. It does prove that such a worldwide study can be done at a relatively low cost, using Web-based tools available at most academic centers. Furthermore, no institutional/government grant funding was requested or required. As for the future, the authors of TPS plan to repeat the PIRST after the next edition of TPS. Before that occurs, some time needs to elapse to allow for the assimilation of the current system and publication of works concerning the system's utility. The data indicate that cytotechnologists performed as well as any group and suggest that a focused study on cytotechnologist accuracy for the sake of signing out NHGUC would be interesting. Finally, the nature of low-grade urothelial neoplasia needs additional investigation. Are the low grade lesions characterized by changes in fibroblast growth factor 3 (FGF-3) capable of invasion, or are the rare examples of invasive low-grade lesions a result of study contamination with the more genetically unstable HGUC characterized by p53 mutation or some sort of hybrid lesion? Publisher Copyright: {\textcopyright} 2018 American Society of Cytopathology",

year = "2018",

month = jul,

day = "1",

doi = "10.1016/j.jasc.2018.02.005",

language = "English (US)",

volume = "7",

pages = "174--184",

journal = "Journal of the American Society of Cytopathology",

issn = "2213-2945",

publisher = "Elsevier BV",

number = "4",

}

TY - JOUR

T1 - Paris Interobserver Reproducibility Study (PIRST)

AU - Kurtycz, Daniel F.I.

AU - Barkan, Güliz A.

AU - Pavelec, Derek M.

AU - Rosenthal, Dorothy L.

AU - Wojcik, Eva M.

AU - VandenBussche, Christopher J.

AU - Mangiulli, Kala

AU - Olson, Matthew T.

N1 - Funding Information: The major findings of this study were: (1) the single most significant feature affecting the performance of survey responders in the application of TPS was the diagnostic category of the image; (2) the highest levels of interobserver reproducibility and accuracy were seen for NHGUC and HGUC; (3) the lowest performance levels were seen in detecting non-urothelial, non-squamous neoplasms and the indeterminate categories AUC and SHGUC; (4) subspecialization in cytopathology showed an increase in diagnostic accuracy among both cytopathologists and cytotechnologists in univariate analysis but not in a multivariate analysis; (5) there was no discernable difference between the performance of cytotechnologists and cytopathologists at classifying images according to TPS; and (6) there was no association between practice type or experience level in the performance of respondents at classifying images according to TPS. A challenge of 85 images to the cytologic community demonstrated that the feature most clearly associated with the performance of morphology is the diagnostic category, with NHGUC and HGUC being the most likely categories to be categorized correctly. This finding is both expected and welcome because it reinforces both practical experience and the universal consensus on the distinct cytomorphology of HGUC as opposed to Low-Grade Urothelial Neoplasia, which has been shown to have poor interobserver variability. 12 The classification of reactive conditions, formerly considered “atypical”, as NHGUC concurrently increased the size of this category and likely increased interobserver reproducibility. This claim is difficult to quantify because there have not been many studies performed prior to TPS on this topic. 13 The extremely poor performance of TPS with recognizing the cytological features of tumors that are neither urothelial nor squamous reflects the morphologic subtlety of these tumors in urinary cytology. No classification system can substitute for a broad differential diagnosis and knowledge of the past medical history leading to an index of suspicion for a non-urothelial malignancy, including metastatic/extravesical lesions. The poor reproducibility of indeterminate categories in cytology classification systems is well documented and TPS is no exception. 8,12,14-16 Thus, the finding that AUC and SHGUC had worse interobserver reproducibility than NHGUC and HGUC was expected. More interesting is the observation that a substantial amount of ambiguity lies around SHGUC, with this category splitting evenly between HGUC and AUC. Although this may not appear desirable prima facie, it is indeed welcome if it represents a shift away from the AUC versus NHGUC boundary. If this is the case, then it is early evidence that emphasis on HGUC and the de-emphasis of Low-Grade Urothelial Neoplasm have moved the point of disagreement among observers to a more clinically actionable space. Reiterating, it is the intent of TPS to separate urothelial cytology into two groups, one negative for high-grade urothelial carcinoma and one worrisome or positive for high-grade urothelial carcinoma. This means that the AUC and SHGUC categories are confined to those samples worrisome for HGUC. In many laboratories it has been common practice to use atypia for an array of inflammatory, infectious, and reactive conditions as well as those cases indeterminate for neoplasia. In TPS this is no longer the case. Excluding all of the former cases and restricting the indeterminate for neoplasia (AUC) category to those with cells having an N:C ratio of 0.5 or higher should lead to decrease in the use of AUC. SHGUC is also relatively restricted and defined as having 5 to 10 non-degenerate, non-superficial cells with N:C ratios of 0.5 to 0.7 and coarse chromatin or irregular membranes. If NHGUC and AUC become clearly separated from one another, the relative risk of AUC for representing carcinoma becomes greater. Thus, the positive predictive value of AUC improves. Because the suspicious category of SHGUC will also not be contaminated by abnormalities less than worrisome for HGUC, its relative risk and positive predictive value also improves. The separation should lead to a greater diagnostic yield of HGUC from cystoscopies following AUC and SHGUC cytology. The TPS authors do believe that, because of decreased interpretations of AUC in cases of GU cytology, there will be fewer patients needing cystoscopy. Alternatively, the reset in risk and predictive value for AUC and SHGUC should lead to improved detection of HGUC for cystoscopies performed on patients with AUC and SHGUC samples. Most respondents replied that that their laboratories receive voided urine (98%), catheterized urine (81%), bladder wash specimens (85%), ureteral washing/brushing specimens (71%), and renal pelvis washing/brushing specimens (68%). This is in line with a previous a College of American Pathologists (CAP) survey where voided urine, bladder washing/barbotage, and catheterized urine specimens were found to represent 99.6%, 86.6%, and 88.5% of the urine samples, respectively. 17 In addition, 745 participants noted that specimens in their laboratories are processed using ThinPrep (64%), cytospin (42%), SurePath (17%), conventional smears (21%), and other preparatory techniques (6%); this is similar to the findings of the CAP survey where ThinPrep (57.4%) and cytospin (45.5%) were found to be the most commonly used preparation types in urinary cytology. 17 Most of the morphological evidence of TPS was based on studies performed on ThinPrep and SurePath samples; however, both of the aforementioned surveys highlight that the ThinPrep and cytospin are the most common preparation types. This will necessitate additional morphological studies to be performed on cytospin preparations prior to an updated publication of TPS. Another problematic area for TPS is adequacy. There is no general agreement in the literature and there is debate on number of cells or volume or a combination of volume and cellularity. Morphologist characteristics such as training, cytology board certification, practice setting, and exposure to urinary cytology samples were all examined in this study to determine the appeal and feasibility of TPS utilization to the community at large. Of these features, only cytopathology subspecialization as documented by cytopathology board certification appeared to portend significantly better interobserver reproducibility in univariate analysis. As expected, there is an intuitive rationale for better performance among board-certified cytopathologists because those individuals are more likely to have practiced by TPS norms—especially pertaining to NHGUC and HGUC—before TPS was officially released. Neither setting nor annual volume of urinary cytology samples appeared to correlate significantly with performance on classifying images according to TPS in any of the subanalyses. The generally accepted view that long experience or high volume improves accuracy is not consistent with the current data; nonetheless, it should be noted that at the time of this study TPS was new and that the study may have been the first exposure of many of the participants to TPS. Therefore, volume and experience may not have helped with concordance. Recent literature also indicates that many cytologists tend to overestimate N:C ratio. 18 On the other hand, a similar lack of concordance with volume and experience was seen in the GYN Bethesda Interobserver reproducibility study. 19 As to why it was the case with Bethesda GYN, there is only speculation. Contrary to TPS, there are decades of experience with the Bethesda GYN categories. In these reproducibility studies, and much of teaching, still images are used. These are not whole slide samples. The still images are not representative of actual clinical specimens in that locator skills are not challenged and are often seen at higher magnification than in clinical practice. There are additional acknowledged limitations to this study. This study was geographically concentrated. The majority of respondents to this online survey were from North America (68%); this is probably a result of the methods used in requesting participation. Although requests went through the Web site of the International Academy of Cytopathology and to societies based in Asia, most of the activity was carried out through the Web site and listserv of the American Society of Cytopathology. Additionally, the survey was in English. Thus, the survey is over-weighted with the North American–English speaking experience. Neither TPS, nor this study, included images relative to radiation change or chemotherapy. Because such situations could prove diagnostically challenging, future editions of TPS and interobserver reproducibility studies need to take these sorts of samples into account. The Paris System Consortium has a number of objectives for refinement of the system and a number of questions to be answered. These are outlined in the afterword of the TPS and provide direction for future research efforts. 7 Goals include defining relative risk of the diagnostic categories; restricting use of atypical (“inconclusive”) categories; examining what actually constitutes an adequate sample for urinary cytology; and assessing how well users apply the criteria of TPS. 20-23 This study helps establish a baseline in support of the latter objective. Of note, the reclassification of images was not one of the aims of this study. The authors intended to formally identify the less-reproducible categories and set the groundwork for investigations that will hopefully lead to improved clarity. TPS aims to progressively refine the definitions for the sake of better diagnosis. Changes in the images used for future editions of TPS will depend on studies investigating utility of the categories, the risk of malignancy for a given category, and evolving photography. An important aspect of this project was that this survey was performed solely by volunteer effort with help from the American Society of Cytopathology Central Office and the support of the Division of Information Technology of the University of Wisconsin. It does prove that such a worldwide study can be done at a relatively low cost, using Web-based tools available at most academic centers. Furthermore, no institutional/government grant funding was requested or required. As for the future, the authors of TPS plan to repeat the PIRST after the next edition of TPS. Before that occurs, some time needs to elapse to allow for the assimilation of the current system and publication of works concerning the system's utility. The data indicate that cytotechnologists performed as well as any group and suggest that a focused study on cytotechnologist accuracy for the sake of signing out NHGUC would be interesting. Finally, the nature of low-grade urothelial neoplasia needs additional investigation. Are the low grade lesions characterized by changes in fibroblast growth factor 3 (FGF-3) capable of invasion, or are the rare examples of invasive low-grade lesions a result of study contamination with the more genetically unstable HGUC characterized by p53 mutation or some sort of hybrid lesion? Publisher Copyright: © 2018 American Society of Cytopathology

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Objectives: In concert with the 2015 publication of The Paris System for Urinary Cytopathology (TPS), a Web-based interobserver study, co-sponsored by the American Society of Cytopathology (ASC) and International Academy of Cytology (IAC), was performed to determine diagnostic agreement among volunteer participants and with the TPS author consensus. Material and Methods: Participants at various levels of training and certification were recruited through national and international cytopathology professional societies. Although the survey was open to all comers, potential participants were screened by two basic cytopathology questions. Information was collected on the level of training, practice patterns, and experience. Study participants evaluated 85 images (previously unpublished) chosen from the TPS atlas. These images spanned all diagnostic categories. Results: Of the 1993 attempts to access the survey, 1313 participants correctly answered the qualifying questions and were included in the survey. Respondents were concentrated in the United States, although many participants came from other countries. The majority of respondents were board-certified in anatomic pathology with cytopathology certification. A smaller number were cytotechnologists. Board-certified cytopathologists and specialist cytotechnologists outperformed other certifications. Practice type (academics versus non-academic), and country (US versus international) were not major factors in concordance. Diagnostic categories with the best agreement were Negative for High-Grade Urothelial Carcinoma (NHGUC; 71%), Low-Grade Urothelial Neoplasm (LGUN; 62%), and High-Grade Urothelial Carcinoma (HGUC; 57%). Indeterminate categories showed low concordance. Conclusions: The NHGUC, LGUN, and HGUC were most correlated with diagnostic agreement among observers. This study can serve as a baseline for future comparisons.

AB - Objectives: In concert with the 2015 publication of The Paris System for Urinary Cytopathology (TPS), a Web-based interobserver study, co-sponsored by the American Society of Cytopathology (ASC) and International Academy of Cytology (IAC), was performed to determine diagnostic agreement among volunteer participants and with the TPS author consensus. Material and Methods: Participants at various levels of training and certification were recruited through national and international cytopathology professional societies. Although the survey was open to all comers, potential participants were screened by two basic cytopathology questions. Information was collected on the level of training, practice patterns, and experience. Study participants evaluated 85 images (previously unpublished) chosen from the TPS atlas. These images spanned all diagnostic categories. Results: Of the 1993 attempts to access the survey, 1313 participants correctly answered the qualifying questions and were included in the survey. Respondents were concentrated in the United States, although many participants came from other countries. The majority of respondents were board-certified in anatomic pathology with cytopathology certification. A smaller number were cytotechnologists. Board-certified cytopathologists and specialist cytotechnologists outperformed other certifications. Practice type (academics versus non-academic), and country (US versus international) were not major factors in concordance. Diagnostic categories with the best agreement were Negative for High-Grade Urothelial Carcinoma (NHGUC; 71%), Low-Grade Urothelial Neoplasm (LGUN; 62%), and High-Grade Urothelial Carcinoma (HGUC; 57%). Indeterminate categories showed low concordance. Conclusions: The NHGUC, LGUN, and HGUC were most correlated with diagnostic agreement among observers. This study can serve as a baseline for future comparisons.

KW - Bladder cancer

KW - Interobserver reproducibility

KW - Standardized Terminology System

KW - The Paris System

KW - Urine cytology

KW - Web Based Image Study

UR - http://www.scopus.com/inward/record.url?scp=85044948018&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044948018&partnerID=8YFLogxK

U2 - 10.1016/j.jasc.2018.02.005

DO - 10.1016/j.jasc.2018.02.005

M3 - Article

C2 - 31043274

AN - SCOPUS:85044948018

VL - 7

SP - 174

EP - 184

JO - Journal of the American Society of Cytopathology

JF - Journal of the American Society of Cytopathology

SN - 2213-2945

IS - 4

ER -

Paris Interobserver Reproducibility Study (PIRST)

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this