TY - JOUR
T1 - Code and Data Sharing Practices in the Radiology Artificial Intelligence Literature
T2 - A Meta-Research Study
AU - Venkatesh, Kesavan
AU - Santomartino, Samantha M.
AU - Sulam, Jeremias
AU - Yi, Paul H.
N1 - Publisher Copyright:
© RSNA, 2022.
PY - 2022/9
Y1 - 2022/9
N2 - Purpose: To evaluate code and data sharing practices in original artificial intelligence (AI) scientific manuscripts published in the Radiological Society of North America (RSNA) journals suite from 2017 through 2021. Materials and Methods: A retrospective meta-research study was conducted of articles published in the RSNA journals suite from January 1, 2017, through December 31, 2021. A total of 218 articles were included and evaluated for code sharing practices, reproducibility of shared code, and data sharing practices. Categorical comparisons were conducted using Fisher exact tests with respect to year and journal of publication, author affiliation(s), and type of algorithm used. Results: Of the 218 included articles, 73 (34%) shared code, with 24 (33% of code sharing articles and 11% of all articles) sharing reproducible code. Radiology and Radiology: Artificial Intelligence published the most code sharing articles (48 [66%] and 21 [29%], respectively). Twenty-nine articles (13%) shared data, and 12 of these articles (41% of data sharing articles) shared complete experimental data by using only public domain datasets. Four of the 218 articles (2%) shared both code and complete experimental data. Code sharing rates were statistically higher in 2020 and 2021 compared with earlier years (P,.01) and were higher in Radiology and Radiology: Artificial Intelligence compared with other journals (P,.01). Conclusion: Original AI scientific articles in the RSNA journals suite had low rates of code and data sharing, emphasizing the need for open-source code and data to achieve transparent and reproducible science.
AB - Purpose: To evaluate code and data sharing practices in original artificial intelligence (AI) scientific manuscripts published in the Radiological Society of North America (RSNA) journals suite from 2017 through 2021. Materials and Methods: A retrospective meta-research study was conducted of articles published in the RSNA journals suite from January 1, 2017, through December 31, 2021. A total of 218 articles were included and evaluated for code sharing practices, reproducibility of shared code, and data sharing practices. Categorical comparisons were conducted using Fisher exact tests with respect to year and journal of publication, author affiliation(s), and type of algorithm used. Results: Of the 218 included articles, 73 (34%) shared code, with 24 (33% of code sharing articles and 11% of all articles) sharing reproducible code. Radiology and Radiology: Artificial Intelligence published the most code sharing articles (48 [66%] and 21 [29%], respectively). Twenty-nine articles (13%) shared data, and 12 of these articles (41% of data sharing articles) shared complete experimental data by using only public domain datasets. Four of the 218 articles (2%) shared both code and complete experimental data. Code sharing rates were statistically higher in 2020 and 2021 compared with earlier years (P,.01) and were higher in Radiology and Radiology: Artificial Intelligence compared with other journals (P,.01). Conclusion: Original AI scientific articles in the RSNA journals suite had low rates of code and data sharing, emphasizing the need for open-source code and data to achieve transparent and reproducible science.
KW - AI in Education
KW - Machine Learning
KW - Meta-Analysis
UR - http://www.scopus.com/inward/record.url?scp=85139136866&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85139136866&partnerID=8YFLogxK
U2 - 10.1148/ryai.220081
DO - 10.1148/ryai.220081
M3 - Article
C2 - 36204536
AN - SCOPUS:85139136866
SN - 2638-6100
VL - 4
JO - Radiology: Artificial Intelligence
JF - Radiology: Artificial Intelligence
IS - 5
M1 - e220081
ER -