Interobserver agreement in assessment of clinical variables in children with blunt head trauma

Objectives: To be useful in development of clinical decision rules, clinical variables must demonstrate acceptable agreement when assessed by different observers. The objective was to determine the interobserver agreement in the assessment of historical and physical examination findings of children undergoing emergency department (ED) evaluation for blunt head trauma. Methods: This was a prospective cohort study of children younger than 18 years evaluated for blunt head trauma at one of 25 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Patients were excluded if injury occurred more than 24 hours prior to evaluation, if neuroimaging was obtained at another hospital prior to evaluation, or if the patient had a clinically trivial mechanism of injury. Two clinicians independently completed a standardized clinical assessment on a templated data form. Assessments were performed within 60 minutes of each other and prior to clinician review of any neuroimaging (if obtained). Agreement between the two observers beyond that expected by chance was calculated for each clinical variable, using the kappa (j) statistic for categorical variables and weighted kappa for ordinal variables. Variables with a lower 95% confidence limit (LCL) of κ > 0.4 were considered to have acceptable agreement. Results: Fifteen-hundred pairs of observations were obtained. Acceptable agreement was achieved in 27 of the 32 variables studied (84%). Mechanism of injury (low, medium, or high risk) had κ = 0.83. For subjective symptoms, kappa ranged from 0.47 (dizziness) to 0.93 (frequency of vomiting); all had 95% LCL > 0.4. Of the physical examination findings, kappa ranged from 0.22 (agitated) to 0.89 (Glasgow Coma Scale [GCS] score). The 95% LCL for kappa was <0.4 for four individual signs of altered mental status and for quality (i.e., boggy or firm) of scalp hematoma if present. Conclusions: Both subjective and objective clinical variables in children with blunt head trauma can be assessed by different observers with acceptable agreement, making these variables suitable candidates for clinical decision rules.