Quantitative assessment of structural image quality

Adon F.G. Rosen, David R. Roalf, Kosha Ruparel, Jason Blake, Kevin Seelaus, Lakshmi P. Villa, Rastko Ciric, Philip A. Cook, Christos Davatzikos, Mark A. Elliott, Angel Garcia de La Garza, Efstathios D. Gennatas, Megan Quarmley, J. Eric Schmitt, Russell T. Shinohara, M. Dylan Tisdall, R. Cameron Craddock, Raquel E. Gur, Ruben C. Gur, Theodore D. Satterthwaite

Research output: Contribution to journalArticle

Abstract

Data quality is increasingly recognized as one of the most important confounding factors in brain imaging research. It is particularly important for studies of brain development, where age is systematically related to in-scanner motion and data quality. Prior work has demonstrated that in-scanner head motion biases estimates of structural neuroimaging measures. However, objective measures of data quality are not available for most structural brain images. Here we sought to identify quantitative measures of data quality for T1-weighted volumes, describe how these measures relate to cortical thickness, and delineate how this in turn may bias inference regarding associations with age in youth. Three highly-trained raters provided manual ratings of 1840 raw T1-weighted volumes. These images included a training set of 1065 images from Philadelphia Neurodevelopmental Cohort (PNC), a test set of 533 images from the PNC, as well as an external test set of 242 adults acquired on a different scanner. Manual ratings were compared to automated quality measures provided by the Preprocessed Connectomes Project's Quality Assurance Protocol (QAP), as well as FreeSurfer's Euler number, which summarizes the topological complexity of the reconstructed cortical surface. Results revealed that the Euler number was consistently correlated with manual ratings across samples. Furthermore, the Euler number could be used to identify images scored “unusable” by human raters with a high degree of accuracy (AUC: 0.98–0.99), and out-performed proxy measures from functional timeseries acquired in the same scanning session. The Euler number also was significantly related to cortical thickness in a regionally heterogeneous pattern that was consistent across datasets and replicated prior results. Finally, data quality both inflated and obscured associations with age during adolescence. Taken together, these results indicate that reliable measures of data quality can be automatically derived from T1-weighted volumes, and that failing to control for data quality can systematically bias the results of studies of brain maturation.

Original languageEnglish (US)
Pages (from-to)407-418
Number of pages12
JournalNeuroImage
Volume169
DOIs
StatePublished - Apr 1 2018

Fingerprint

Neuroimaging
Brain
Connectome
Proxy
Data Accuracy
Area Under Curve
Head
Research
Datasets

Keywords

  • Artifact
  • Development
  • Motion
  • MRI
  • Structural imaging
  • T1

ASJC Scopus subject areas

  • Neurology
  • Cognitive Neuroscience

Cite this

Rosen, A. F. G., Roalf, D. R., Ruparel, K., Blake, J., Seelaus, K., Villa, L. P., ... Satterthwaite, T. D. (2018). Quantitative assessment of structural image quality. NeuroImage, 169, 407-418. https://doi.org/10.1016/j.neuroimage.2017.12.059

Quantitative assessment of structural image quality. / Rosen, Adon F.G.; Roalf, David R.; Ruparel, Kosha; Blake, Jason; Seelaus, Kevin; Villa, Lakshmi P.; Ciric, Rastko; Cook, Philip A.; Davatzikos, Christos; Elliott, Mark A.; Garcia de La Garza, Angel; Gennatas, Efstathios D.; Quarmley, Megan; Schmitt, J. Eric; Shinohara, Russell T.; Tisdall, M. Dylan; Craddock, R. Cameron; Gur, Raquel E.; Gur, Ruben C.; Satterthwaite, Theodore D.

In: NeuroImage, Vol. 169, 01.04.2018, p. 407-418.

Research output: Contribution to journalArticle

Rosen, AFG, Roalf, DR, Ruparel, K, Blake, J, Seelaus, K, Villa, LP, Ciric, R, Cook, PA, Davatzikos, C, Elliott, MA, Garcia de La Garza, A, Gennatas, ED, Quarmley, M, Schmitt, JE, Shinohara, RT, Tisdall, MD, Craddock, RC, Gur, RE, Gur, RC & Satterthwaite, TD 2018, 'Quantitative assessment of structural image quality', NeuroImage, vol. 169, pp. 407-418. https://doi.org/10.1016/j.neuroimage.2017.12.059
Rosen AFG, Roalf DR, Ruparel K, Blake J, Seelaus K, Villa LP et al. Quantitative assessment of structural image quality. NeuroImage. 2018 Apr 1;169:407-418. https://doi.org/10.1016/j.neuroimage.2017.12.059
Rosen, Adon F.G. ; Roalf, David R. ; Ruparel, Kosha ; Blake, Jason ; Seelaus, Kevin ; Villa, Lakshmi P. ; Ciric, Rastko ; Cook, Philip A. ; Davatzikos, Christos ; Elliott, Mark A. ; Garcia de La Garza, Angel ; Gennatas, Efstathios D. ; Quarmley, Megan ; Schmitt, J. Eric ; Shinohara, Russell T. ; Tisdall, M. Dylan ; Craddock, R. Cameron ; Gur, Raquel E. ; Gur, Ruben C. ; Satterthwaite, Theodore D. / Quantitative assessment of structural image quality. In: NeuroImage. 2018 ; Vol. 169. pp. 407-418.
@article{8c146d0ef0114f37a12c3a716b0d6d17,
title = "Quantitative assessment of structural image quality",
abstract = "Data quality is increasingly recognized as one of the most important confounding factors in brain imaging research. It is particularly important for studies of brain development, where age is systematically related to in-scanner motion and data quality. Prior work has demonstrated that in-scanner head motion biases estimates of structural neuroimaging measures. However, objective measures of data quality are not available for most structural brain images. Here we sought to identify quantitative measures of data quality for T1-weighted volumes, describe how these measures relate to cortical thickness, and delineate how this in turn may bias inference regarding associations with age in youth. Three highly-trained raters provided manual ratings of 1840 raw T1-weighted volumes. These images included a training set of 1065 images from Philadelphia Neurodevelopmental Cohort (PNC), a test set of 533 images from the PNC, as well as an external test set of 242 adults acquired on a different scanner. Manual ratings were compared to automated quality measures provided by the Preprocessed Connectomes Project's Quality Assurance Protocol (QAP), as well as FreeSurfer's Euler number, which summarizes the topological complexity of the reconstructed cortical surface. Results revealed that the Euler number was consistently correlated with manual ratings across samples. Furthermore, the Euler number could be used to identify images scored “unusable” by human raters with a high degree of accuracy (AUC: 0.98–0.99), and out-performed proxy measures from functional timeseries acquired in the same scanning session. The Euler number also was significantly related to cortical thickness in a regionally heterogeneous pattern that was consistent across datasets and replicated prior results. Finally, data quality both inflated and obscured associations with age during adolescence. Taken together, these results indicate that reliable measures of data quality can be automatically derived from T1-weighted volumes, and that failing to control for data quality can systematically bias the results of studies of brain maturation.",
keywords = "Artifact, Development, Motion, MRI, Structural imaging, T1",
author = "Rosen, {Adon F.G.} and Roalf, {David R.} and Kosha Ruparel and Jason Blake and Kevin Seelaus and Villa, {Lakshmi P.} and Rastko Ciric and Cook, {Philip A.} and Christos Davatzikos and Elliott, {Mark A.} and {Garcia de La Garza}, Angel and Gennatas, {Efstathios D.} and Megan Quarmley and Schmitt, {J. Eric} and Shinohara, {Russell T.} and Tisdall, {M. Dylan} and Craddock, {R. Cameron} and Gur, {Raquel E.} and Gur, {Ruben C.} and Satterthwaite, {Theodore D.}",
year = "2018",
month = "4",
day = "1",
doi = "10.1016/j.neuroimage.2017.12.059",
language = "English (US)",
volume = "169",
pages = "407--418",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Quantitative assessment of structural image quality

AU - Rosen, Adon F.G.

AU - Roalf, David R.

AU - Ruparel, Kosha

AU - Blake, Jason

AU - Seelaus, Kevin

AU - Villa, Lakshmi P.

AU - Ciric, Rastko

AU - Cook, Philip A.

AU - Davatzikos, Christos

AU - Elliott, Mark A.

AU - Garcia de La Garza, Angel

AU - Gennatas, Efstathios D.

AU - Quarmley, Megan

AU - Schmitt, J. Eric

AU - Shinohara, Russell T.

AU - Tisdall, M. Dylan

AU - Craddock, R. Cameron

AU - Gur, Raquel E.

AU - Gur, Ruben C.

AU - Satterthwaite, Theodore D.

PY - 2018/4/1

Y1 - 2018/4/1

N2 - Data quality is increasingly recognized as one of the most important confounding factors in brain imaging research. It is particularly important for studies of brain development, where age is systematically related to in-scanner motion and data quality. Prior work has demonstrated that in-scanner head motion biases estimates of structural neuroimaging measures. However, objective measures of data quality are not available for most structural brain images. Here we sought to identify quantitative measures of data quality for T1-weighted volumes, describe how these measures relate to cortical thickness, and delineate how this in turn may bias inference regarding associations with age in youth. Three highly-trained raters provided manual ratings of 1840 raw T1-weighted volumes. These images included a training set of 1065 images from Philadelphia Neurodevelopmental Cohort (PNC), a test set of 533 images from the PNC, as well as an external test set of 242 adults acquired on a different scanner. Manual ratings were compared to automated quality measures provided by the Preprocessed Connectomes Project's Quality Assurance Protocol (QAP), as well as FreeSurfer's Euler number, which summarizes the topological complexity of the reconstructed cortical surface. Results revealed that the Euler number was consistently correlated with manual ratings across samples. Furthermore, the Euler number could be used to identify images scored “unusable” by human raters with a high degree of accuracy (AUC: 0.98–0.99), and out-performed proxy measures from functional timeseries acquired in the same scanning session. The Euler number also was significantly related to cortical thickness in a regionally heterogeneous pattern that was consistent across datasets and replicated prior results. Finally, data quality both inflated and obscured associations with age during adolescence. Taken together, these results indicate that reliable measures of data quality can be automatically derived from T1-weighted volumes, and that failing to control for data quality can systematically bias the results of studies of brain maturation.

AB - Data quality is increasingly recognized as one of the most important confounding factors in brain imaging research. It is particularly important for studies of brain development, where age is systematically related to in-scanner motion and data quality. Prior work has demonstrated that in-scanner head motion biases estimates of structural neuroimaging measures. However, objective measures of data quality are not available for most structural brain images. Here we sought to identify quantitative measures of data quality for T1-weighted volumes, describe how these measures relate to cortical thickness, and delineate how this in turn may bias inference regarding associations with age in youth. Three highly-trained raters provided manual ratings of 1840 raw T1-weighted volumes. These images included a training set of 1065 images from Philadelphia Neurodevelopmental Cohort (PNC), a test set of 533 images from the PNC, as well as an external test set of 242 adults acquired on a different scanner. Manual ratings were compared to automated quality measures provided by the Preprocessed Connectomes Project's Quality Assurance Protocol (QAP), as well as FreeSurfer's Euler number, which summarizes the topological complexity of the reconstructed cortical surface. Results revealed that the Euler number was consistently correlated with manual ratings across samples. Furthermore, the Euler number could be used to identify images scored “unusable” by human raters with a high degree of accuracy (AUC: 0.98–0.99), and out-performed proxy measures from functional timeseries acquired in the same scanning session. The Euler number also was significantly related to cortical thickness in a regionally heterogeneous pattern that was consistent across datasets and replicated prior results. Finally, data quality both inflated and obscured associations with age during adolescence. Taken together, these results indicate that reliable measures of data quality can be automatically derived from T1-weighted volumes, and that failing to control for data quality can systematically bias the results of studies of brain maturation.

KW - Artifact

KW - Development

KW - Motion

KW - MRI

KW - Structural imaging

KW - T1

UR - http://www.scopus.com/inward/record.url?scp=85039785972&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85039785972&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2017.12.059

DO - 10.1016/j.neuroimage.2017.12.059

M3 - Article

VL - 169

SP - 407

EP - 418

JO - NeuroImage

JF - NeuroImage

SN - 1053-8119

ER -