TY - JOUR
T1 - Evaluation of prostate segmentation algorithms for MRI
T2 - The PROMISE12 challenge
AU - Litjens, Geert
AU - Toth, Robert
AU - van de Ven, Wendy
AU - Hoeks, Caroline
AU - Kerkstra, Sjoerd
AU - van Ginneken, Bram
AU - Vincent, Graham
AU - Guillard, Gwenael
AU - Birbeck, Neil
AU - Zhang, Jindang
AU - Strand, Robin
AU - Malmberg, Filip
AU - Ou, Yangming
AU - Davatzikos, Christos
AU - Kirschner, Matthias
AU - Jung, Florian
AU - Yuan, Jing
AU - Qiu, Wu
AU - Gao, Qinquan
AU - Edwards, Philip Eddie
AU - Maan, Bianca
AU - van der Heijden, Ferdinand
AU - Ghose, Soumya
AU - Mitra, Jhimli
AU - Dowling, Jason
AU - Barratt, Dean
AU - Huisman, Henkjan
AU - Madabhushi, Anant
N1 - Funding Information:
This research was funded by Grant KUN2007-3971 from the Dutch Cancer Society and by the National Cancer Institute of the National Institutes of Health under Award Nos. R01CA136535-01, R01CA140772-01, and R21CA167811-01; the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under Award No. R43EB015199-01; the National Science Foundation under Award No. IIP-1248316; the QED award from the University City Science Center and Rutgers University. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
PY - 2014/2
Y1 - 2014/2
N2 - Prostate MRI image segmentation has been an area of intense research due to the increased use of MRI as a modality for the clinical workup of prostate cancer. Segmentation is useful for various tasks, e.g. to accurately localize prostate boundaries for radiotherapy or to initialize multi-modal registration algorithms. In the past, it has been difficult for research groups to evaluate prostate segmentation algorithms on multi-center, multi-vendor and multi-protocol data. Especially because we are dealing with MR images, image appearance, resolution and the presence of artifacts are affected by differences in scanners and/or protocols, which in turn can have a large influence on algorithm accuracy. The Prostate MR Image Segmentation (PROMISE12) challenge was setup to allow a fair and meaningful comparison of segmentation methods on the basis of performance and robustness. In this work we will discuss the initial results of the online PROMISE12 challenge, and the results obtained in the live challenge workshop hosted by the MICCAI2012 conference. In the challenge, 100 prostate MR cases from 4 different centers were included, with differences in scanner manufacturer, field strength and protocol. A total of 11 teams from academic research groups and industry participated. Algorithms showed a wide variety in methods and implementation, including active appearance models, atlas registration and level sets. Evaluation was performed using boundary and volume based metrics which were combined into a single score relating the metrics to human expert performance. The winners of the challenge where the algorithms by teams Imorphics and ScrAutoProstate, with scores of 85.72 and 84.29 overall. Both algorithms where significantly better than all other algorithms in the challenge (p < 0.05) and had an efficient implementation with a run time of 8. min and 3. s per case respectively. Overall, active appearance model based approaches seemed to outperform other approaches like multi-atlas registration, both on accuracy and computation time. Although average algorithm performance was good to excellent and the Imorphics algorithm outperformed the second observer on average, we showed that algorithm combination might lead to further improvement, indicating that optimal performance for prostate segmentation is not yet obtained. All results are available online at http://promise12.grand-challenge.org/.
AB - Prostate MRI image segmentation has been an area of intense research due to the increased use of MRI as a modality for the clinical workup of prostate cancer. Segmentation is useful for various tasks, e.g. to accurately localize prostate boundaries for radiotherapy or to initialize multi-modal registration algorithms. In the past, it has been difficult for research groups to evaluate prostate segmentation algorithms on multi-center, multi-vendor and multi-protocol data. Especially because we are dealing with MR images, image appearance, resolution and the presence of artifacts are affected by differences in scanners and/or protocols, which in turn can have a large influence on algorithm accuracy. The Prostate MR Image Segmentation (PROMISE12) challenge was setup to allow a fair and meaningful comparison of segmentation methods on the basis of performance and robustness. In this work we will discuss the initial results of the online PROMISE12 challenge, and the results obtained in the live challenge workshop hosted by the MICCAI2012 conference. In the challenge, 100 prostate MR cases from 4 different centers were included, with differences in scanner manufacturer, field strength and protocol. A total of 11 teams from academic research groups and industry participated. Algorithms showed a wide variety in methods and implementation, including active appearance models, atlas registration and level sets. Evaluation was performed using boundary and volume based metrics which were combined into a single score relating the metrics to human expert performance. The winners of the challenge where the algorithms by teams Imorphics and ScrAutoProstate, with scores of 85.72 and 84.29 overall. Both algorithms where significantly better than all other algorithms in the challenge (p < 0.05) and had an efficient implementation with a run time of 8. min and 3. s per case respectively. Overall, active appearance model based approaches seemed to outperform other approaches like multi-atlas registration, both on accuracy and computation time. Although average algorithm performance was good to excellent and the Imorphics algorithm outperformed the second observer on average, we showed that algorithm combination might lead to further improvement, indicating that optimal performance for prostate segmentation is not yet obtained. All results are available online at http://promise12.grand-challenge.org/.
KW - Challenge
KW - MRI
KW - Prostate
KW - Segmentation
UR - http://www.scopus.com/inward/record.url?scp=84892426437&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84892426437&partnerID=8YFLogxK
U2 - 10.1016/j.media.2013.12.002
DO - 10.1016/j.media.2013.12.002
M3 - Article
C2 - 24418598
AN - SCOPUS:84892426437
SN - 1361-8415
VL - 18
SP - 359
EP - 373
JO - Medical image analysis
JF - Medical image analysis
IS - 2
ER -