Abstract
Assessment of attentional processes via computerized assessment is frequently used to quantify intra-individual cognitive improvement or decline in response to treatment. However, assessment of intra-individual change is highly dependent on sufficient test reliability. We examined the test-retest reliability of selected variables from one popular computerized continuous performance test (CPT)i.e., the Conners' CPT - Second Edition (CPT-II). Participants were 39 healthy children (20 girls) ages 6-18 without intellectual impairment (mean PPVT-III SS = 102.6), LD, or psychiatric disorders (DICA-IV). Test-retest reliability over the 3-8 month interval (mean = 6 months) was acceptable (Intraclass Correlations [ICC] =.82 to.92) on comparison measures (Beery Test of Visual Perception, WISC-IV Block Design, PPVT-III). In contrast, test-retest reliability was only modest for CPT-II raw scores (ICCs ranging from.62 to.82) and T-scores (ICCs ranging from.33 to.65) for variables of interest (Omissions, Commissions, Variability, Hit Reaction Time, and Attentiveness). Using test-retest reliability information published in the CPT-II manual, 90% confidence intervals based on reliable change index (RCI) methodology were constructed to examine the significance of test-retest difference/change scores. Of the participants in this sample of typically developing youth, 30% generated intra-individual changes in T-scores on the Omissions and Attentiveness variables that exceeded the 90% confidence intervals and qualified as statistically rare changes in score. These results suggest a considerable degree of normal variability in CPT-II test scores over extended test-retest intervals, and suggest a need for caution when interpreting test score changes in neurologically unstable clinical populations.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 1213-1231 |
| Number of pages | 19 |
| Journal | Clinical Neuropsychologist |
| Volume | 23 |
| Issue number | 7 |
| DOIs | |
| State | Published - Sep 2009 |
Fingerprint
Keywords
- Attention
- Conners' Continuous Performance Test
- Neuropsychology
- Normal development
- Reliable Change Index
- Serial assessment
ASJC Scopus subject areas
- Psychiatry and Mental health
- Neuropsychology and Physiological Psychology
- Clinical Psychology
Cite this
Reliability concerns in the repeated computerized assessment of attention in children. / Zabel, Theodore A; Von Thomsen, Christian; Cole, Carolyn; Martin, Rebecca; Mahone, Ernest M.
In: Clinical Neuropsychologist, Vol. 23, No. 7, 09.2009, p. 1213-1231.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Reliability concerns in the repeated computerized assessment of attention in children
AU - Zabel, Theodore A
AU - Von Thomsen, Christian
AU - Cole, Carolyn
AU - Martin, Rebecca
AU - Mahone, Ernest M
PY - 2009/9
Y1 - 2009/9
N2 - Assessment of attentional processes via computerized assessment is frequently used to quantify intra-individual cognitive improvement or decline in response to treatment. However, assessment of intra-individual change is highly dependent on sufficient test reliability. We examined the test-retest reliability of selected variables from one popular computerized continuous performance test (CPT)i.e., the Conners' CPT - Second Edition (CPT-II). Participants were 39 healthy children (20 girls) ages 6-18 without intellectual impairment (mean PPVT-III SS = 102.6), LD, or psychiatric disorders (DICA-IV). Test-retest reliability over the 3-8 month interval (mean = 6 months) was acceptable (Intraclass Correlations [ICC] =.82 to.92) on comparison measures (Beery Test of Visual Perception, WISC-IV Block Design, PPVT-III). In contrast, test-retest reliability was only modest for CPT-II raw scores (ICCs ranging from.62 to.82) and T-scores (ICCs ranging from.33 to.65) for variables of interest (Omissions, Commissions, Variability, Hit Reaction Time, and Attentiveness). Using test-retest reliability information published in the CPT-II manual, 90% confidence intervals based on reliable change index (RCI) methodology were constructed to examine the significance of test-retest difference/change scores. Of the participants in this sample of typically developing youth, 30% generated intra-individual changes in T-scores on the Omissions and Attentiveness variables that exceeded the 90% confidence intervals and qualified as statistically rare changes in score. These results suggest a considerable degree of normal variability in CPT-II test scores over extended test-retest intervals, and suggest a need for caution when interpreting test score changes in neurologically unstable clinical populations.
AB - Assessment of attentional processes via computerized assessment is frequently used to quantify intra-individual cognitive improvement or decline in response to treatment. However, assessment of intra-individual change is highly dependent on sufficient test reliability. We examined the test-retest reliability of selected variables from one popular computerized continuous performance test (CPT)i.e., the Conners' CPT - Second Edition (CPT-II). Participants were 39 healthy children (20 girls) ages 6-18 without intellectual impairment (mean PPVT-III SS = 102.6), LD, or psychiatric disorders (DICA-IV). Test-retest reliability over the 3-8 month interval (mean = 6 months) was acceptable (Intraclass Correlations [ICC] =.82 to.92) on comparison measures (Beery Test of Visual Perception, WISC-IV Block Design, PPVT-III). In contrast, test-retest reliability was only modest for CPT-II raw scores (ICCs ranging from.62 to.82) and T-scores (ICCs ranging from.33 to.65) for variables of interest (Omissions, Commissions, Variability, Hit Reaction Time, and Attentiveness). Using test-retest reliability information published in the CPT-II manual, 90% confidence intervals based on reliable change index (RCI) methodology were constructed to examine the significance of test-retest difference/change scores. Of the participants in this sample of typically developing youth, 30% generated intra-individual changes in T-scores on the Omissions and Attentiveness variables that exceeded the 90% confidence intervals and qualified as statistically rare changes in score. These results suggest a considerable degree of normal variability in CPT-II test scores over extended test-retest intervals, and suggest a need for caution when interpreting test score changes in neurologically unstable clinical populations.
KW - Attention
KW - Conners' Continuous Performance Test
KW - Neuropsychology
KW - Normal development
KW - Reliable Change Index
KW - Serial assessment
UR - http://www.scopus.com/inward/record.url?scp=70349449658&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=70349449658&partnerID=8YFLogxK
U2 - 10.1080/13854040902855358
DO - 10.1080/13854040902855358
M3 - Article
VL - 23
SP - 1213
EP - 1231
JO - Clinical Neuropsychologist
JF - Clinical Neuropsychologist
SN - 1385-4046
IS - 7
ER -