Comparing test-retest reliability of dynamic functional connectivity methods

Ann S. Choe, Mary Beth Nebel, Anita D. Barber, Jessica R. Cohen, Yuting Xu, James J. Pekar, Brian Caffo, Martin A. Lindquist

Research output: Contribution to journalArticle

Abstract

Due to the dynamic, condition-dependent nature of brain activity, interest in estimating rapid functional connectivity (FC) changes that occur during resting-state functional magnetic resonance imaging (rs-fMRI) has recently soared. However, studying dynamic FC is methodologically challenging, due to the low signal-to-noise ratio of the blood oxygen level dependent (BOLD) signal in fMRI and the massive number of data points generated during the analysis. Thus, it is important to establish methods and summary measures that maximize reliability and the utility of dynamic FC to provide insight into brain function. In this study, we investigated the reliability of dynamic FC summary measures derived using three commonly used estimation methods - sliding window (SW), tapered sliding window (TSW), and dynamic conditional correlations (DCC) methods. We applied each of these techniques to two publicly available rs-fMRI test-retest data sets - the Multi-Modal MRI Reproducibility Resource (Kirby Data) and the Human Connectome Project (HCP Data). The reliability of two categories of dynamic FC summary measures were assessed, specifically basic summary statistics of the dynamic correlations and summary measures derived from recurring whole-brain patterns of FC (“brain states”). The results provide evidence that dynamic correlations are reliably detected in both test-retest data sets, and the DCC method outperforms SW methods in terms of the reliability of summary statistics. However, across all estimation methods, reliability of the brain state-derived measures was low. Notably, the results also show that the DCC-derived dynamic correlation variances are significantly more reliable than those derived using the non-parametric estimation methods. This is important, as the fluctuations of dynamic FC (i.e., its variance) has a strong potential to provide summary measures that can be used to find meaningful individual differences in dynamic FC. We therefore conclude that utilizing the variance of the dynamic connectivity is an important component in any dynamic FC-derived summary measure.

Original languageEnglish (US)
Pages (from-to)155-175
Number of pages21
JournalNeuroImage
Volume158
DOIs
StatePublished - Sep 1 2017

Fingerprint

Brain
Magnetic Resonance Imaging
Datasets
Connectome
Signal-To-Noise Ratio
Individuality
Oxygen

ASJC Scopus subject areas

  • Neurology
  • Cognitive Neuroscience

Cite this

Comparing test-retest reliability of dynamic functional connectivity methods. / Choe, Ann S.; Nebel, Mary Beth; Barber, Anita D.; Cohen, Jessica R.; Xu, Yuting; Pekar, James J.; Caffo, Brian; Lindquist, Martin A.

In: NeuroImage, Vol. 158, 01.09.2017, p. 155-175.

Research output: Contribution to journalArticle

Choe AS, Nebel MB, Barber AD, Cohen JR, Xu Y, Pekar JJ et al. Comparing test-retest reliability of dynamic functional connectivity methods. NeuroImage. 2017 Sep 1;158:155-175. Available from, DOI: 10.1016/j.neuroimage.2017.07.005

Choe, Ann S.; Nebel, Mary Beth; Barber, Anita D.; Cohen, Jessica R.; Xu, Yuting; Pekar, James J.; Caffo, Brian; Lindquist, Martin A. / Comparing test-retest reliability of dynamic functional connectivity methods.

In: NeuroImage, Vol. 158, 01.09.2017, p. 155-175.

Research output: Contribution to journalArticle

@article{5b6bdc8c0f564627b6903ab7072fdcb0,
title = "Comparing test-retest reliability of dynamic functional connectivity methods",
abstract = "Due to the dynamic, condition-dependent nature of brain activity, interest in estimating rapid functional connectivity (FC) changes that occur during resting-state functional magnetic resonance imaging (rs-fMRI) has recently soared. However, studying dynamic FC is methodologically challenging, due to the low signal-to-noise ratio of the blood oxygen level dependent (BOLD) signal in fMRI and the massive number of data points generated during the analysis. Thus, it is important to establish methods and summary measures that maximize reliability and the utility of dynamic FC to provide insight into brain function. In this study, we investigated the reliability of dynamic FC summary measures derived using three commonly used estimation methods - sliding window (SW), tapered sliding window (TSW), and dynamic conditional correlations (DCC) methods. We applied each of these techniques to two publicly available rs-fMRI test-retest data sets - the Multi-Modal MRI Reproducibility Resource (Kirby Data) and the Human Connectome Project (HCP Data). The reliability of two categories of dynamic FC summary measures were assessed, specifically basic summary statistics of the dynamic correlations and summary measures derived from recurring whole-brain patterns of FC (“brain states”). The results provide evidence that dynamic correlations are reliably detected in both test-retest data sets, and the DCC method outperforms SW methods in terms of the reliability of summary statistics. However, across all estimation methods, reliability of the brain state-derived measures was low. Notably, the results also show that the DCC-derived dynamic correlation variances are significantly more reliable than those derived using the non-parametric estimation methods. This is important, as the fluctuations of dynamic FC (i.e., its variance) has a strong potential to provide summary measures that can be used to find meaningful individual differences in dynamic FC. We therefore conclude that utilizing the variance of the dynamic connectivity is an important component in any dynamic FC-derived summary measure.",
author = "Choe, {Ann S.} and Nebel, {Mary Beth} and Barber, {Anita D.} and Cohen, {Jessica R.} and Yuting Xu and Pekar, {James J.} and Brian Caffo and Lindquist, {Martin A.}",
year = "2017",
month = "9",
doi = "10.1016/j.neuroimage.2017.07.005",
volume = "158",
pages = "155--175",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Comparing test-retest reliability of dynamic functional connectivity methods

AU - Choe,Ann S.

AU - Nebel,Mary Beth

AU - Barber,Anita D.

AU - Cohen,Jessica R.

AU - Xu,Yuting

AU - Pekar,James J.

AU - Caffo,Brian

AU - Lindquist,Martin A.

PY - 2017/9/1

Y1 - 2017/9/1

N2 - Due to the dynamic, condition-dependent nature of brain activity, interest in estimating rapid functional connectivity (FC) changes that occur during resting-state functional magnetic resonance imaging (rs-fMRI) has recently soared. However, studying dynamic FC is methodologically challenging, due to the low signal-to-noise ratio of the blood oxygen level dependent (BOLD) signal in fMRI and the massive number of data points generated during the analysis. Thus, it is important to establish methods and summary measures that maximize reliability and the utility of dynamic FC to provide insight into brain function. In this study, we investigated the reliability of dynamic FC summary measures derived using three commonly used estimation methods - sliding window (SW), tapered sliding window (TSW), and dynamic conditional correlations (DCC) methods. We applied each of these techniques to two publicly available rs-fMRI test-retest data sets - the Multi-Modal MRI Reproducibility Resource (Kirby Data) and the Human Connectome Project (HCP Data). The reliability of two categories of dynamic FC summary measures were assessed, specifically basic summary statistics of the dynamic correlations and summary measures derived from recurring whole-brain patterns of FC (“brain states”). The results provide evidence that dynamic correlations are reliably detected in both test-retest data sets, and the DCC method outperforms SW methods in terms of the reliability of summary statistics. However, across all estimation methods, reliability of the brain state-derived measures was low. Notably, the results also show that the DCC-derived dynamic correlation variances are significantly more reliable than those derived using the non-parametric estimation methods. This is important, as the fluctuations of dynamic FC (i.e., its variance) has a strong potential to provide summary measures that can be used to find meaningful individual differences in dynamic FC. We therefore conclude that utilizing the variance of the dynamic connectivity is an important component in any dynamic FC-derived summary measure.

AB - Due to the dynamic, condition-dependent nature of brain activity, interest in estimating rapid functional connectivity (FC) changes that occur during resting-state functional magnetic resonance imaging (rs-fMRI) has recently soared. However, studying dynamic FC is methodologically challenging, due to the low signal-to-noise ratio of the blood oxygen level dependent (BOLD) signal in fMRI and the massive number of data points generated during the analysis. Thus, it is important to establish methods and summary measures that maximize reliability and the utility of dynamic FC to provide insight into brain function. In this study, we investigated the reliability of dynamic FC summary measures derived using three commonly used estimation methods - sliding window (SW), tapered sliding window (TSW), and dynamic conditional correlations (DCC) methods. We applied each of these techniques to two publicly available rs-fMRI test-retest data sets - the Multi-Modal MRI Reproducibility Resource (Kirby Data) and the Human Connectome Project (HCP Data). The reliability of two categories of dynamic FC summary measures were assessed, specifically basic summary statistics of the dynamic correlations and summary measures derived from recurring whole-brain patterns of FC (“brain states”). The results provide evidence that dynamic correlations are reliably detected in both test-retest data sets, and the DCC method outperforms SW methods in terms of the reliability of summary statistics. However, across all estimation methods, reliability of the brain state-derived measures was low. Notably, the results also show that the DCC-derived dynamic correlation variances are significantly more reliable than those derived using the non-parametric estimation methods. This is important, as the fluctuations of dynamic FC (i.e., its variance) has a strong potential to provide summary measures that can be used to find meaningful individual differences in dynamic FC. We therefore conclude that utilizing the variance of the dynamic connectivity is an important component in any dynamic FC-derived summary measure.

UR - http://www.scopus.com/inward/record.url?scp=85022175614&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85022175614&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2017.07.005

DO - 10.1016/j.neuroimage.2017.07.005

M3 - Article

VL - 158

SP - 155

EP - 175

JO - NeuroImage

T2 - NeuroImage

JF - NeuroImage

SN - 1053-8119

ER -