TY - JOUR
T1 - Learning Inductive Attention Guidance for Partially Supervised Pancreatic Ductal Adenocarcinoma Prediction
AU - Wang, Yan
AU - Tang, Peng
AU - Zhou, Yuyin
AU - Shen, Wei
AU - Fishman, Elliot K.
AU - Yuille, Alan L.
N1 - Funding Information:
Manuscript received January 7, 2021; accepted February 9, 2021. Date of publication February 18, 2021; date of current version September 30, 2021. This work was supported by the Lustgarten Foundation for Pancreatic Cancer Research. (Corresponding author: Wei Shen.) Yan Wang is with the Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai 200241, China, and also with the Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218 USA (e-mail: wyanny.9@gmail.com).
Publisher Copyright:
© 1982-2012 IEEE.
PY - 2021/10/1
Y1 - 2021/10/1
N2 - Pancreatic ductal adenocarcinoma (PDAC) is the third most common cause of cancer death in the United States. Predicting tumors like PDACs (including both classification and segmentation) from medical images by deep learning is becoming a growing trend, but usually a large number of annotated data are required for training, which is very labor-intensive and time-consuming. In this paper, we consider a partially supervised setting, where cheap image-level annotations are provided for all the training data, and the costly per-voxel annotations are only available for a subset of them. We propose an Inductive Attention Guidance Network (IAG-Net) to jointly learn a global image-level classifier for normal/PDAC classification and a local voxel-level classifier for semi-supervised PDAC segmentation. We instantiate both the global and the local classifiers by multiple instance learning (MIL), where the attention guidance, indicating roughly where the PDAC regions are, is the key to bridging them: For global MIL based normal/PDAC classification, attention serves as a weight for each instance (voxel) during MIL pooling, which eliminates the distraction from the background; For local MIL based semi-supervised PDAC segmentation, the attention guidance is inductive, which not only provides bag-level pseudo-labels to training data without per-voxel annotations for MIL training, but also acts as a proxy of an instance-level classifier. Experimental results show that our IAG-Net boosts PDAC segmentation accuracy by more than 5% compared with the state-of-the-arts.
AB - Pancreatic ductal adenocarcinoma (PDAC) is the third most common cause of cancer death in the United States. Predicting tumors like PDACs (including both classification and segmentation) from medical images by deep learning is becoming a growing trend, but usually a large number of annotated data are required for training, which is very labor-intensive and time-consuming. In this paper, we consider a partially supervised setting, where cheap image-level annotations are provided for all the training data, and the costly per-voxel annotations are only available for a subset of them. We propose an Inductive Attention Guidance Network (IAG-Net) to jointly learn a global image-level classifier for normal/PDAC classification and a local voxel-level classifier for semi-supervised PDAC segmentation. We instantiate both the global and the local classifiers by multiple instance learning (MIL), where the attention guidance, indicating roughly where the PDAC regions are, is the key to bridging them: For global MIL based normal/PDAC classification, attention serves as a weight for each instance (voxel) during MIL pooling, which eliminates the distraction from the background; For local MIL based semi-supervised PDAC segmentation, the attention guidance is inductive, which not only provides bag-level pseudo-labels to training data without per-voxel annotations for MIL training, but also acts as a proxy of an instance-level classifier. Experimental results show that our IAG-Net boosts PDAC segmentation accuracy by more than 5% compared with the state-of-the-arts.
KW - Attention
KW - medical image segmentation
KW - multiple instance learning
KW - semi-supervised learning
UR - http://www.scopus.com/inward/record.url?scp=85101767402&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85101767402&partnerID=8YFLogxK
U2 - 10.1109/TMI.2021.3060066
DO - 10.1109/TMI.2021.3060066
M3 - Article
C2 - 33600311
AN - SCOPUS:85101767402
VL - 40
SP - 2723
EP - 2735
JO - IEEE Transactions on Medical Imaging
JF - IEEE Transactions on Medical Imaging
SN - 0278-0062
IS - 10
ER -