Comparing test-retest reliability of dynamic functional connectivity methods

Ann Choe, Mary Beth Nebel, Anita D. Barber, Jessica R. Cohen, Yuting Xu, James J Pekar, Brian S Caffo, Martin Lindquist

Research output: Contribution to journalArticle

Abstract

Due to the dynamic, condition-dependent nature of brain activity, interest in estimating rapid functional connectivity (FC) changes that occur during resting-state functional magnetic resonance imaging (rs-fMRI) has recently soared. However, studying dynamic FC is methodologically challenging, due to the low signal-to-noise ratio of the blood oxygen level dependent (BOLD) signal in fMRI and the massive number of data points generated during the analysis. Thus, it is important to establish methods and summary measures that maximize reliability and the utility of dynamic FC to provide insight into brain function. In this study, we investigated the reliability of dynamic FC summary measures derived using three commonly used estimation methods - sliding window (SW), tapered sliding window (TSW), and dynamic conditional correlations (DCC) methods. We applied each of these techniques to two publicly available rs-fMRI test-retest data sets - the Multi-Modal MRI Reproducibility Resource (Kirby Data) and the Human Connectome Project (HCP Data). The reliability of two categories of dynamic FC summary measures were assessed, specifically basic summary statistics of the dynamic correlations and summary measures derived from recurring whole-brain patterns of FC (“brain states”). The results provide evidence that dynamic correlations are reliably detected in both test-retest data sets, and the DCC method outperforms SW methods in terms of the reliability of summary statistics. However, across all estimation methods, reliability of the brain state-derived measures was low. Notably, the results also show that the DCC-derived dynamic correlation variances are significantly more reliable than those derived using the non-parametric estimation methods. This is important, as the fluctuations of dynamic FC (i.e., its variance) has a strong potential to provide summary measures that can be used to find meaningful individual differences in dynamic FC. We therefore conclude that utilizing the variance of the dynamic connectivity is an important component in any dynamic FC-derived summary measure.

Original languageEnglish (US)
Pages (from-to)155-175
Number of pages21
JournalNeuroImage
Volume158
DOIs
StatePublished - Sep 1 2017

Fingerprint

Reproducibility of Results
Brain
Magnetic Resonance Imaging
Connectome
Signal-To-Noise Ratio
Individuality
Oxygen

ASJC Scopus subject areas

  • Neurology
  • Cognitive Neuroscience

Cite this

Comparing test-retest reliability of dynamic functional connectivity methods. / Choe, Ann; Nebel, Mary Beth; Barber, Anita D.; Cohen, Jessica R.; Xu, Yuting; Pekar, James J; Caffo, Brian S; Lindquist, Martin.

In: NeuroImage, Vol. 158, 01.09.2017, p. 155-175.

Research output: Contribution to journalArticle

Choe, Ann ; Nebel, Mary Beth ; Barber, Anita D. ; Cohen, Jessica R. ; Xu, Yuting ; Pekar, James J ; Caffo, Brian S ; Lindquist, Martin. / Comparing test-retest reliability of dynamic functional connectivity methods. In: NeuroImage. 2017 ; Vol. 158. pp. 155-175.
@article{5b6bdc8c0f564627b6903ab7072fdcb0,
title = "Comparing test-retest reliability of dynamic functional connectivity methods",
abstract = "Due to the dynamic, condition-dependent nature of brain activity, interest in estimating rapid functional connectivity (FC) changes that occur during resting-state functional magnetic resonance imaging (rs-fMRI) has recently soared. However, studying dynamic FC is methodologically challenging, due to the low signal-to-noise ratio of the blood oxygen level dependent (BOLD) signal in fMRI and the massive number of data points generated during the analysis. Thus, it is important to establish methods and summary measures that maximize reliability and the utility of dynamic FC to provide insight into brain function. In this study, we investigated the reliability of dynamic FC summary measures derived using three commonly used estimation methods - sliding window (SW), tapered sliding window (TSW), and dynamic conditional correlations (DCC) methods. We applied each of these techniques to two publicly available rs-fMRI test-retest data sets - the Multi-Modal MRI Reproducibility Resource (Kirby Data) and the Human Connectome Project (HCP Data). The reliability of two categories of dynamic FC summary measures were assessed, specifically basic summary statistics of the dynamic correlations and summary measures derived from recurring whole-brain patterns of FC (“brain states”). The results provide evidence that dynamic correlations are reliably detected in both test-retest data sets, and the DCC method outperforms SW methods in terms of the reliability of summary statistics. However, across all estimation methods, reliability of the brain state-derived measures was low. Notably, the results also show that the DCC-derived dynamic correlation variances are significantly more reliable than those derived using the non-parametric estimation methods. This is important, as the fluctuations of dynamic FC (i.e., its variance) has a strong potential to provide summary measures that can be used to find meaningful individual differences in dynamic FC. We therefore conclude that utilizing the variance of the dynamic connectivity is an important component in any dynamic FC-derived summary measure.",
author = "Ann Choe and Nebel, {Mary Beth} and Barber, {Anita D.} and Cohen, {Jessica R.} and Yuting Xu and Pekar, {James J} and Caffo, {Brian S} and Martin Lindquist",
year = "2017",
month = "9",
day = "1",
doi = "10.1016/j.neuroimage.2017.07.005",
language = "English (US)",
volume = "158",
pages = "155--175",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Comparing test-retest reliability of dynamic functional connectivity methods

AU - Choe, Ann

AU - Nebel, Mary Beth

AU - Barber, Anita D.

AU - Cohen, Jessica R.

AU - Xu, Yuting

AU - Pekar, James J

AU - Caffo, Brian S

AU - Lindquist, Martin

PY - 2017/9/1

Y1 - 2017/9/1

N2 - Due to the dynamic, condition-dependent nature of brain activity, interest in estimating rapid functional connectivity (FC) changes that occur during resting-state functional magnetic resonance imaging (rs-fMRI) has recently soared. However, studying dynamic FC is methodologically challenging, due to the low signal-to-noise ratio of the blood oxygen level dependent (BOLD) signal in fMRI and the massive number of data points generated during the analysis. Thus, it is important to establish methods and summary measures that maximize reliability and the utility of dynamic FC to provide insight into brain function. In this study, we investigated the reliability of dynamic FC summary measures derived using three commonly used estimation methods - sliding window (SW), tapered sliding window (TSW), and dynamic conditional correlations (DCC) methods. We applied each of these techniques to two publicly available rs-fMRI test-retest data sets - the Multi-Modal MRI Reproducibility Resource (Kirby Data) and the Human Connectome Project (HCP Data). The reliability of two categories of dynamic FC summary measures were assessed, specifically basic summary statistics of the dynamic correlations and summary measures derived from recurring whole-brain patterns of FC (“brain states”). The results provide evidence that dynamic correlations are reliably detected in both test-retest data sets, and the DCC method outperforms SW methods in terms of the reliability of summary statistics. However, across all estimation methods, reliability of the brain state-derived measures was low. Notably, the results also show that the DCC-derived dynamic correlation variances are significantly more reliable than those derived using the non-parametric estimation methods. This is important, as the fluctuations of dynamic FC (i.e., its variance) has a strong potential to provide summary measures that can be used to find meaningful individual differences in dynamic FC. We therefore conclude that utilizing the variance of the dynamic connectivity is an important component in any dynamic FC-derived summary measure.

AB - Due to the dynamic, condition-dependent nature of brain activity, interest in estimating rapid functional connectivity (FC) changes that occur during resting-state functional magnetic resonance imaging (rs-fMRI) has recently soared. However, studying dynamic FC is methodologically challenging, due to the low signal-to-noise ratio of the blood oxygen level dependent (BOLD) signal in fMRI and the massive number of data points generated during the analysis. Thus, it is important to establish methods and summary measures that maximize reliability and the utility of dynamic FC to provide insight into brain function. In this study, we investigated the reliability of dynamic FC summary measures derived using three commonly used estimation methods - sliding window (SW), tapered sliding window (TSW), and dynamic conditional correlations (DCC) methods. We applied each of these techniques to two publicly available rs-fMRI test-retest data sets - the Multi-Modal MRI Reproducibility Resource (Kirby Data) and the Human Connectome Project (HCP Data). The reliability of two categories of dynamic FC summary measures were assessed, specifically basic summary statistics of the dynamic correlations and summary measures derived from recurring whole-brain patterns of FC (“brain states”). The results provide evidence that dynamic correlations are reliably detected in both test-retest data sets, and the DCC method outperforms SW methods in terms of the reliability of summary statistics. However, across all estimation methods, reliability of the brain state-derived measures was low. Notably, the results also show that the DCC-derived dynamic correlation variances are significantly more reliable than those derived using the non-parametric estimation methods. This is important, as the fluctuations of dynamic FC (i.e., its variance) has a strong potential to provide summary measures that can be used to find meaningful individual differences in dynamic FC. We therefore conclude that utilizing the variance of the dynamic connectivity is an important component in any dynamic FC-derived summary measure.

UR - http://www.scopus.com/inward/record.url?scp=85022175614&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85022175614&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2017.07.005

DO - 10.1016/j.neuroimage.2017.07.005

M3 - Article

C2 - 28687517

AN - SCOPUS:85022175614

VL - 158

SP - 155

EP - 175

JO - NeuroImage

JF - NeuroImage

SN - 1053-8119

ER -