Cross-cohort heterogeneity encountered while validating a model for HIV disease progression among antiretroviral initiators

Objective: To evaluate a model for predicting time to AIDS or death among HIV-infected persons initiating highly active antiretroviral therapy (HAART). Study Design and Setting: The model was constructed from 1,891 HAART initiators in the Collaborations in HIV Outcomes Research/US (CHORUS) cohort. The model's predictive ability was assessed using internal bootstrap validation techniques and data from 716 HAART initiators at Johns Hopkins HIV Clinical Cohort (JHHCC) in whom HIV disease was, in general, more advanced. Results: The estimated concordance statistic was 0.632 with the bootstrap method and 0.625 in JHHCC. Mean predicted and observed 3-year AIDS-free survival for JHHCC was 0.76 and 0.73 (95% confidence interval [CI], 0.69-0.77), respectively; mean predicted and observed 5-year AIDS-free survival was 0.69 and 0.57 (95% CI, 0.52-0.62), respectively. Sensitivity analyses showed that the discrepancy between predicted and observed AIDS-free survival after 3 years could be due to differences in lost-to-follow-up rates between cohorts. Conclusion: The model was fair at using baseline characteristics to order patients' risk of disease progression, but did not accurately predict AIDS-free survival >3 years after HAART initiation. Different variable definitions, patient characteristics, and loss to follow-up highlight the challenges of using data from one cohort to predict AIDS-free survival in an independent cohort.