DSM-5 field trials in the United States and Canada, part II: Test-retest reliability of selected categorical diagnoses

Objective: The DSM-5 Field Trials were designed to obtain precise (standard error <0.1) estimates of the intraclass kappa as a measure of the degree to which two clinicians could independently agree on the presence or absence of selected DSM-5 diagnoses when the same patient was interviewed on separate occasions, in clinical settings, and evaluated with usual clinical interview methods. Method: Eleven academic centers in the United States and Canada were selected, and each was assigned several target diagnoses frequently treated in that setting. Consecutive patients visiting a site during the study were screened and stratified on the basis of DSM-IV diagnoses or symptomatic presentations. Patients were randomly assigned to two clinicians for a diagnostic interview; clinicians were blind to any previous diagnosis. All data were entered directly via an Internet-based software system to a secure central server. Detailed research design and statistical methods are presented in an accompanying article. Results: There were a total of 15 adult and eight child/adolescent diagnoses for which adequate sample sizes were obtained to report adequately precise estimates of the intraclass kappa. Overall, five diagnoses were in the very good range (kappa=0.60-0.79), nine in the good range (kappa=0.40-0.59), six in the questionable range (kappa=0.20-0.39), and three in the unacceptable range (kappa values <0.20). Eight diagnoses had insufficient sample sizes to generate precise kappa estimates at any site. Conclusions: Most diagnoses adequately tested had good to very good reliability with these representative clinical populations assessed with usual clinical interview methods. Some diagnoses that were revised to encompass a broader spectrum of symptom expression or had a more dimensional approach tested in the good to very good range.