Fusion of fMRI, sMRI, and EEG data using canonical correlation analysis

Nicolle M. Correa, Yi Ou Li, Tülay Adali, Vince D. Calhoun

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Typically data acquired through imaging techniques such as functional magnetic resonance imaging (fMRI), structural MRI (sMRI), and electroencephalography (EEG) are analyzed separately. Each modality records brain structure and function at different scales, and fusing information from such complementary modalities promises to provide additional insight into connectivity across brain networks and changes due to disease. Recently, a number of methods have been proposed for data integration and fusion of two brain imaging modalities. We propose a new data fusion scheme based on canonical correlation analysis that enables the detection of associations across multiple modalities. Our multimodal canonical correlation analysis (mCCA) scheme works at the feature level using multi-set CCA to determine inter-subject covariations across modalities. We apply mCCA to fMRI, sMRI, and EEG data collected from patients diagnosed with schizophrenia and healthy controls. Through data collected from an auditory oddball task, we show that the fusion of multiple modalities detects more specific associations as compared to fusion of two modalities.

Original languageEnglish (US)
Title of host publication2009 IEEE International Conference on Acoustics, Speech, and Signal Processing - Proceedings, ICASSP 2009
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages385-388
Number of pages4
ISBN (Print)9781424423545
DOIs
StatePublished - Jan 1 2009
Externally publishedYes
Event2009 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2009 - Taipei, Taiwan, Province of China
Duration: Apr 19 2009Apr 24 2009

Publication series

NameICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
ISSN (Print)1520-6149

Other

Other2009 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2009
CountryTaiwan, Province of China
CityTaipei
Period4/19/094/24/09

Keywords

  • Biomedical fusion
  • Canonical correlation analysis
  • EEG
  • fMRI
  • sMRI

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Fusion of fMRI, sMRI, and EEG data using canonical correlation analysis'. Together they form a unique fingerprint.

Cite this