Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI

Nicolle M. Correa, Tom Eichele, Tülay Adali, Yi Ou Li, Vince Daniel Calhoun

Research output: Contribution to journalArticle

Abstract

Functional magnetic resonance imaging (fMRI) data and electroencephalography (EEG) data provide complementary spatio-temporal information about brain function. Methods to couple the relative strengths of these modalities usually involve two stages: first forming a feature set from each dataset based on one criterion followed by exploration of connections among the features using a second criterion. We propose a data fusion method for simultaneously acquired fMRI and EEG data that combines these steps using a single criterion for finding the cross-modality associations and performing source separation. Using multi-set canonical correlation analysis (M-CCA), we obtain a decomposition of the two modalities, into spatial maps for fMRI data and a corresponding temporal evolution for EEG data, based on trial-to-trial covariation across the two modalities. Additionally, the analysis is performed on data from a group of subjects in order to make group inferences about the covariation across modalities. Being multivariate, the proposed method facilitates the study of brain connectivity along with localization of brain function. M-CCA can be easily extended to incorporate different data types and additional modalities. We demonstrate the promise of the proposed method in finding covarying trial-to-trial amplitude modulations (AMs) in an auditory task involving implicit pattern learning. The results show approximately linear decreasing trends in AMs for both modalities and the corresponding spatial activations occur mainly in motor, frontal, temporal, inferior parietal, and orbito-frontal areas that are linked both to sensory function as well as learning and expectation-all of which match activations related to the presented paradigm.

Original languageEnglish (US)
Pages (from-to)1438-1445
Number of pages8
JournalNeuroImage
Volume50
Issue number4
DOIs
Publication statusPublished - May 1 2010
Externally publishedYes

    Fingerprint

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Neurology

Cite this