Parallel but not equivalent: Challenges and solutions for repeated assessment of cognition over time

Alden L Gross, Sharon K. Inouye, George Rebok, Jason Brandt, Paul K. Crane, Jeanine Parisi, Doug Tommet, Karen J Bandeen Roche, Michelle C Carlson, Richard N. Jones

Research output: Contribution to journalArticle

Abstract

Objective. Analyses of individual differences in change may be unintentionally biased when versions of a neuropsychological test used at different follow-ups are not of equivalent difficulty. This study's objective was to compare mean, linear, and equipercentile equating methods and demonstrate their utility in longitudinal research. Study design and setting: The Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE, N=1,401) study is a longitudinal randomized trial of cognitive training. The Alzheimer's Disease Neuroimaging Initiative (ADNI, n=819) is an observational cohort study. Nonequivalent alternate versions of the Auditory Verbal Learning Test (AVLT) were administered in both studies. Results. Using visual displays, raw and mean-equated AVLT scores in both studies showed obvious nonlinear trajectories in reference groups that should show minimal change and poor equivalence over time (ps.001), and raw scores demonstrated poor fits in models of within-person change (root mean square errors of approximation, RMSEAs > 0.12). Linear and equipercentile equating produced more similar means in reference groups (ps.09) and performed better in growth models (RMSEAs <0.05). Conclusion. Equipercentile equating is the preferred equating method because it accommodates tests more difficult than a reference test at different percentiles of performance and performs well in models of within-person trajectory. The method has broad applications in both clinical and research settings to enhance the ability to use nonequivalent test forms.

Original languageEnglish (US)
Pages (from-to)758-772
Number of pages15
JournalJournal of Clinical and Experimental Neuropsychology
Volume34
Issue number7
DOIs
StatePublished - Aug 1 2012

Fingerprint

Cognition
Verbal Learning
Aptitude
Neuropsychological Tests
Research
Individuality
Neuroimaging
Observational Studies
Alzheimer Disease
Cohort Studies
Growth

Keywords

  • Alternate forms
  • Equating
  • Equipercentile
  • Longitudinal analysis
  • Neuropsychology
  • Parallel forms

ASJC Scopus subject areas

  • Clinical Neurology
  • Neurology
  • Clinical Psychology

Cite this

@article{16cbe094d5f34b8ba187d8c1e5048c19,
title = "Parallel but not equivalent: Challenges and solutions for repeated assessment of cognition over time",
abstract = "Objective. Analyses of individual differences in change may be unintentionally biased when versions of a neuropsychological test used at different follow-ups are not of equivalent difficulty. This study's objective was to compare mean, linear, and equipercentile equating methods and demonstrate their utility in longitudinal research. Study design and setting: The Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE, N=1,401) study is a longitudinal randomized trial of cognitive training. The Alzheimer's Disease Neuroimaging Initiative (ADNI, n=819) is an observational cohort study. Nonequivalent alternate versions of the Auditory Verbal Learning Test (AVLT) were administered in both studies. Results. Using visual displays, raw and mean-equated AVLT scores in both studies showed obvious nonlinear trajectories in reference groups that should show minimal change and poor equivalence over time (ps.001), and raw scores demonstrated poor fits in models of within-person change (root mean square errors of approximation, RMSEAs > 0.12). Linear and equipercentile equating produced more similar means in reference groups (ps.09) and performed better in growth models (RMSEAs <0.05). Conclusion. Equipercentile equating is the preferred equating method because it accommodates tests more difficult than a reference test at different percentiles of performance and performs well in models of within-person trajectory. The method has broad applications in both clinical and research settings to enhance the ability to use nonequivalent test forms.",
keywords = "Alternate forms, Equating, Equipercentile, Longitudinal analysis, Neuropsychology, Parallel forms",
author = "Gross, {Alden L} and Inouye, {Sharon K.} and George Rebok and Jason Brandt and Crane, {Paul K.} and Jeanine Parisi and Doug Tommet and {Bandeen Roche}, {Karen J} and Carlson, {Michelle C} and Jones, {Richard N.}",
year = "2012",
month = "8",
day = "1",
doi = "10.1080/13803395.2012.681628",
language = "English (US)",
volume = "34",
pages = "758--772",
journal = "Journal of Clinical and Experimental Neuropsychology",
issn = "1380-3395",
publisher = "Psychology Press Ltd",
number = "7",

}

TY - JOUR

T1 - Parallel but not equivalent

T2 - Challenges and solutions for repeated assessment of cognition over time

AU - Gross, Alden L

AU - Inouye, Sharon K.

AU - Rebok, George

AU - Brandt, Jason

AU - Crane, Paul K.

AU - Parisi, Jeanine

AU - Tommet, Doug

AU - Bandeen Roche, Karen J

AU - Carlson, Michelle C

AU - Jones, Richard N.

PY - 2012/8/1

Y1 - 2012/8/1

N2 - Objective. Analyses of individual differences in change may be unintentionally biased when versions of a neuropsychological test used at different follow-ups are not of equivalent difficulty. This study's objective was to compare mean, linear, and equipercentile equating methods and demonstrate their utility in longitudinal research. Study design and setting: The Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE, N=1,401) study is a longitudinal randomized trial of cognitive training. The Alzheimer's Disease Neuroimaging Initiative (ADNI, n=819) is an observational cohort study. Nonequivalent alternate versions of the Auditory Verbal Learning Test (AVLT) were administered in both studies. Results. Using visual displays, raw and mean-equated AVLT scores in both studies showed obvious nonlinear trajectories in reference groups that should show minimal change and poor equivalence over time (ps.001), and raw scores demonstrated poor fits in models of within-person change (root mean square errors of approximation, RMSEAs > 0.12). Linear and equipercentile equating produced more similar means in reference groups (ps.09) and performed better in growth models (RMSEAs <0.05). Conclusion. Equipercentile equating is the preferred equating method because it accommodates tests more difficult than a reference test at different percentiles of performance and performs well in models of within-person trajectory. The method has broad applications in both clinical and research settings to enhance the ability to use nonequivalent test forms.

AB - Objective. Analyses of individual differences in change may be unintentionally biased when versions of a neuropsychological test used at different follow-ups are not of equivalent difficulty. This study's objective was to compare mean, linear, and equipercentile equating methods and demonstrate their utility in longitudinal research. Study design and setting: The Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE, N=1,401) study is a longitudinal randomized trial of cognitive training. The Alzheimer's Disease Neuroimaging Initiative (ADNI, n=819) is an observational cohort study. Nonequivalent alternate versions of the Auditory Verbal Learning Test (AVLT) were administered in both studies. Results. Using visual displays, raw and mean-equated AVLT scores in both studies showed obvious nonlinear trajectories in reference groups that should show minimal change and poor equivalence over time (ps.001), and raw scores demonstrated poor fits in models of within-person change (root mean square errors of approximation, RMSEAs > 0.12). Linear and equipercentile equating produced more similar means in reference groups (ps.09) and performed better in growth models (RMSEAs <0.05). Conclusion. Equipercentile equating is the preferred equating method because it accommodates tests more difficult than a reference test at different percentiles of performance and performs well in models of within-person trajectory. The method has broad applications in both clinical and research settings to enhance the ability to use nonequivalent test forms.

KW - Alternate forms

KW - Equating

KW - Equipercentile

KW - Longitudinal analysis

KW - Neuropsychology

KW - Parallel forms

UR - http://www.scopus.com/inward/record.url?scp=84864972337&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84864972337&partnerID=8YFLogxK

U2 - 10.1080/13803395.2012.681628

DO - 10.1080/13803395.2012.681628

M3 - Article

C2 - 22540849

AN - SCOPUS:84864972337

VL - 34

SP - 758

EP - 772

JO - Journal of Clinical and Experimental Neuropsychology

JF - Journal of Clinical and Experimental Neuropsychology

SN - 1380-3395

IS - 7

ER -