DSM-5 field trials in the United States and Canada, part II: Test-retest reliability of selected categorical diagnoses

Darrel A. Regier, William Narrow, Diana E. Clarke, Helena C. Kraemer, S. Janet Kuramoto, Emily A. Kuhl, David J. Kupfer

Research output: Contribution to journalArticle

Abstract

Objective: The DSM-5 Field Trials were designed to obtain precise (standard error <0.1) estimates of the intraclass kappa as a measure of the degree to which two clinicians could independently agree on the presence or absence of selected DSM-5 diagnoses when the same patient was interviewed on separate occasions, in clinical settings, and evaluated with usual clinical interview methods. Method: Eleven academic centers in the United States and Canada were selected, and each was assigned several target diagnoses frequently treated in that setting. Consecutive patients visiting a site during the study were screened and stratified on the basis of DSM-IV diagnoses or symptomatic presentations. Patients were randomly assigned to two clinicians for a diagnostic interview; clinicians were blind to any previous diagnosis. All data were entered directly via an Internet-based software system to a secure central server. Detailed research design and statistical methods are presented in an accompanying article. Results: There were a total of 15 adult and eight child/adolescent diagnoses for which adequate sample sizes were obtained to report adequately precise estimates of the intraclass kappa. Overall, five diagnoses were in the very good range (kappa=0.60-0.79), nine in the good range (kappa=0.40-0.59), six in the questionable range (kappa=0.20-0.39), and three in the unacceptable range (kappa values <0.20). Eight diagnoses had insufficient sample sizes to generate precise kappa estimates at any site. Conclusions: Most diagnoses adequately tested had good to very good reliability with these representative clinical populations assessed with usual clinical interview methods. Some diagnoses that were revised to encompass a broader spectrum of symptom expression or had a more dimensional approach tested in the good to very good range.

Original languageEnglish (US)
Pages (from-to)59-70
Number of pages12
JournalAmerican Journal of Psychiatry
Volume170
Issue number1
DOIs
StatePublished - Jan 1 2013
Externally publishedYes

Fingerprint

Reproducibility of Results
Canada
Interviews
Sample Size
Diagnostic and Statistical Manual of Mental Disorders
Internet
Research Design
Software
Population

ASJC Scopus subject areas

  • Psychiatry and Mental health

Cite this

DSM-5 field trials in the United States and Canada, part II : Test-retest reliability of selected categorical diagnoses. / Regier, Darrel A.; Narrow, William; Clarke, Diana E.; Kraemer, Helena C.; Kuramoto, S. Janet; Kuhl, Emily A.; Kupfer, David J.

In: American Journal of Psychiatry, Vol. 170, No. 1, 01.01.2013, p. 59-70.

Research output: Contribution to journalArticle

Regier, Darrel A. ; Narrow, William ; Clarke, Diana E. ; Kraemer, Helena C. ; Kuramoto, S. Janet ; Kuhl, Emily A. ; Kupfer, David J. / DSM-5 field trials in the United States and Canada, part II : Test-retest reliability of selected categorical diagnoses. In: American Journal of Psychiatry. 2013 ; Vol. 170, No. 1. pp. 59-70.
@article{feff6835ea04448b82d3bae1d3d1dcb4,
title = "DSM-5 field trials in the United States and Canada, part II: Test-retest reliability of selected categorical diagnoses",
abstract = "Objective: The DSM-5 Field Trials were designed to obtain precise (standard error <0.1) estimates of the intraclass kappa as a measure of the degree to which two clinicians could independently agree on the presence or absence of selected DSM-5 diagnoses when the same patient was interviewed on separate occasions, in clinical settings, and evaluated with usual clinical interview methods. Method: Eleven academic centers in the United States and Canada were selected, and each was assigned several target diagnoses frequently treated in that setting. Consecutive patients visiting a site during the study were screened and stratified on the basis of DSM-IV diagnoses or symptomatic presentations. Patients were randomly assigned to two clinicians for a diagnostic interview; clinicians were blind to any previous diagnosis. All data were entered directly via an Internet-based software system to a secure central server. Detailed research design and statistical methods are presented in an accompanying article. Results: There were a total of 15 adult and eight child/adolescent diagnoses for which adequate sample sizes were obtained to report adequately precise estimates of the intraclass kappa. Overall, five diagnoses were in the very good range (kappa=0.60-0.79), nine in the good range (kappa=0.40-0.59), six in the questionable range (kappa=0.20-0.39), and three in the unacceptable range (kappa values <0.20). Eight diagnoses had insufficient sample sizes to generate precise kappa estimates at any site. Conclusions: Most diagnoses adequately tested had good to very good reliability with these representative clinical populations assessed with usual clinical interview methods. Some diagnoses that were revised to encompass a broader spectrum of symptom expression or had a more dimensional approach tested in the good to very good range.",
author = "Regier, {Darrel A.} and William Narrow and Clarke, {Diana E.} and Kraemer, {Helena C.} and Kuramoto, {S. Janet} and Kuhl, {Emily A.} and Kupfer, {David J.}",
year = "2013",
month = "1",
day = "1",
doi = "10.1176/appi.ajp.2012.12070999",
language = "English (US)",
volume = "170",
pages = "59--70",
journal = "American Journal of Psychiatry",
issn = "0002-953X",
publisher = "American Psychiatric Association",
number = "1",

}

TY - JOUR

T1 - DSM-5 field trials in the United States and Canada, part II

T2 - Test-retest reliability of selected categorical diagnoses

AU - Regier, Darrel A.

AU - Narrow, William

AU - Clarke, Diana E.

AU - Kraemer, Helena C.

AU - Kuramoto, S. Janet

AU - Kuhl, Emily A.

AU - Kupfer, David J.

PY - 2013/1/1

Y1 - 2013/1/1

N2 - Objective: The DSM-5 Field Trials were designed to obtain precise (standard error <0.1) estimates of the intraclass kappa as a measure of the degree to which two clinicians could independently agree on the presence or absence of selected DSM-5 diagnoses when the same patient was interviewed on separate occasions, in clinical settings, and evaluated with usual clinical interview methods. Method: Eleven academic centers in the United States and Canada were selected, and each was assigned several target diagnoses frequently treated in that setting. Consecutive patients visiting a site during the study were screened and stratified on the basis of DSM-IV diagnoses or symptomatic presentations. Patients were randomly assigned to two clinicians for a diagnostic interview; clinicians were blind to any previous diagnosis. All data were entered directly via an Internet-based software system to a secure central server. Detailed research design and statistical methods are presented in an accompanying article. Results: There were a total of 15 adult and eight child/adolescent diagnoses for which adequate sample sizes were obtained to report adequately precise estimates of the intraclass kappa. Overall, five diagnoses were in the very good range (kappa=0.60-0.79), nine in the good range (kappa=0.40-0.59), six in the questionable range (kappa=0.20-0.39), and three in the unacceptable range (kappa values <0.20). Eight diagnoses had insufficient sample sizes to generate precise kappa estimates at any site. Conclusions: Most diagnoses adequately tested had good to very good reliability with these representative clinical populations assessed with usual clinical interview methods. Some diagnoses that were revised to encompass a broader spectrum of symptom expression or had a more dimensional approach tested in the good to very good range.

AB - Objective: The DSM-5 Field Trials were designed to obtain precise (standard error <0.1) estimates of the intraclass kappa as a measure of the degree to which two clinicians could independently agree on the presence or absence of selected DSM-5 diagnoses when the same patient was interviewed on separate occasions, in clinical settings, and evaluated with usual clinical interview methods. Method: Eleven academic centers in the United States and Canada were selected, and each was assigned several target diagnoses frequently treated in that setting. Consecutive patients visiting a site during the study were screened and stratified on the basis of DSM-IV diagnoses or symptomatic presentations. Patients were randomly assigned to two clinicians for a diagnostic interview; clinicians were blind to any previous diagnosis. All data were entered directly via an Internet-based software system to a secure central server. Detailed research design and statistical methods are presented in an accompanying article. Results: There were a total of 15 adult and eight child/adolescent diagnoses for which adequate sample sizes were obtained to report adequately precise estimates of the intraclass kappa. Overall, five diagnoses were in the very good range (kappa=0.60-0.79), nine in the good range (kappa=0.40-0.59), six in the questionable range (kappa=0.20-0.39), and three in the unacceptable range (kappa values <0.20). Eight diagnoses had insufficient sample sizes to generate precise kappa estimates at any site. Conclusions: Most diagnoses adequately tested had good to very good reliability with these representative clinical populations assessed with usual clinical interview methods. Some diagnoses that were revised to encompass a broader spectrum of symptom expression or had a more dimensional approach tested in the good to very good range.

UR - http://www.scopus.com/inward/record.url?scp=84872078563&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84872078563&partnerID=8YFLogxK

U2 - 10.1176/appi.ajp.2012.12070999

DO - 10.1176/appi.ajp.2012.12070999

M3 - Article

C2 - 23111466

AN - SCOPUS:84872078563

VL - 170

SP - 59

EP - 70

JO - American Journal of Psychiatry

JF - American Journal of Psychiatry

SN - 0002-953X

IS - 1

ER -