Comparing the Effects of Sampling Designs on the Diagnostic Accuracy of Eight Promising Screening Algorithms for Pediatric Bipolar Disorder

Background: Past studies have used markedly different inclusion and exclusion criteria to form samples used to evaluate diagnostic tests, making it difficult to compare results across studies. The present investigation compared eight screening algorithms in the same sample but under two different design strategies. Methods: The DSM-IV diagnoses were based on a semi-structured diagnostic interview (KSADS) with the parent and youth sequentially. Raters were blind to index test scores. Participants were 216 youths with bipolar spectrum diagnoses and 284 youths with other Axis I diagnoses or no diagnosis. T-tests evaluated whether areas under the curve (AUC) from receiver operating characteristic analyses differed under the two design conditions. Results: All of the instruments discriminated bipolar cases better when inclusion and exclusion criteria duplicated those used in phenomenological research studies selecting narrow phenotypic cases (AUCs ranging from .90 to .81). The measures performed less well when more heterogeneous clinical presentations were included [AUCs ranging from .86 to .69, t(8) = 4.99, p = 001]. Conclusions: Results indicate that checklists perform less well discriminating pediatric bipolar disorder under conditions that more closely resemble clinical practice. Test users must consider whether the sampling strategies and participant characteristics used to evaluate tests match the characteristics of their own patients.