A Multi-task Deep Learning Framework to Localize the Eloquent Cortex in Brain Tumor Patients Using Dynamic Functional Connectivity

Naresh Nandakumar, Niharika Shimona D’Souza, Komal Manzoor, Jay J. Pillai, Sachin K. Gujar, Haris I. Sair, Archana Venkataraman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a novel deep learning framework that uses dynamic functional connectivity to simultaneously localize the language and motor areas of the eloquent cortex in brain tumor patients. Our method leverages convolutional layers to extract graph-based features from the dynamic connectivity matrices and a long-short term memory (LSTM) attention network to weight the relevant time points during classification. The final stage of our model employs multi-task learning to identify different eloquent subsystems. Our unique training strategy finds a shared representation between the cognitive networks of interest, which enables us to handle missing patient data. We evaluate our method on resting-state fMRI data from 56 brain tumor patients while using task fMRI activations as surrogate ground-truth labels for training and testing. Our model achieves higher localization accuracies than conventional deep learning approaches and can identify bilateral language areas even when trained on left-hemisphere lateralized cases. Hence, our method may ultimately be useful for preoperative mapping in tumor patients.

Original languageEnglish (US)
Title of host publicationMachine Learning in Clinical Neuroimaging and Radiogenomics in Neuro-oncology - 3rd International Workshop, MLCN 2020, and Second International Workshop, RNO-AI 2020, Held in Conjunction with MICCAI 2020, Proceedings
EditorsSeyed Mostafa Kia, Hassan Mohy-ud-Din, Ahmed Abdulkadir, Cher Bass, Mohamad Habes, Jane Maryam Rondina, Chantal Tax, Hongzhi Wang, Thomas Wolfers, Saima Rathore, Madhura Ingalhalikar
PublisherSpringer Science and Business Media Deutschland GmbH
Pages34-44
Number of pages11
ISBN (Print)9783030668426
DOIs
StatePublished - 2020
Event3rd International Workshop on Machine Learning in Clinical Neuroimaging, MLCN 2020, and 2nd International Workshop on Radiogenomics in Neuro-oncology, RNO-AI 2020, held in conjunction with MICCAI 2020 - Lima, Peru
Duration: Oct 4 2020Oct 8 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12449 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference3rd International Workshop on Machine Learning in Clinical Neuroimaging, MLCN 2020, and 2nd International Workshop on Radiogenomics in Neuro-oncology, RNO-AI 2020, held in conjunction with MICCAI 2020
CountryPeru
CityLima
Period10/4/2010/8/20

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'A Multi-task Deep Learning Framework to Localize the Eloquent Cortex in Brain Tumor Patients Using Dynamic Functional Connectivity'. Together they form a unique fingerprint.

Cite this