Improving the quality of discrete-choice experiments in health: how can we assess validity and reliability?

Ellen M. Janssen, Deborah A. Marshall, A. Brett Hauber, John F.P. Bridges

Research output: Contribution to journalReview article


Introduction: The recent endorsement of discrete-choice experiments (DCEs) and other stated-preference methods by regulatory and health technology assessment (HTA) agencies has placed a greater focus on demonstrating the validity and reliability of preference results. Areas covered: We present a practical overview of tests of validity and reliability that have been applied in the health DCE literature and explore other study qualities of DCEs. From the published literature, we identify a variety of methods to assess the validity and reliability of DCEs. We conceptualize these methods to create a conceptual model with four domains: measurement validity, measurement reliability, choice validity, and choice reliability. Each domain consists of three categories that can be assessed using one to four procedures (for a total of 24 tests). We present how these tests have been applied in the literature and direct readers to applications of these tests in the health DCE literature. Based on a stakeholder engagement exercise, we consider the importance of study characteristics beyond traditional concepts of validity and reliability. Expert commentary: We discuss study design considerations to assess the validity and reliability of a DCE, consider limitations to the current application of tests, and discuss future work to consider the quality of DCEs in healthcare.

Original languageEnglish (US)
Pages (from-to)531-542
Number of pages12
JournalExpert Review of Pharmacoeconomics and Outcomes Research
Issue number6
Publication statusPublished - Nov 2 2017



  • Benefit-risk assessment
  • discrete-choice experiment
  • patient preferences
  • reliability
  • stated-preference methods
  • validity

ASJC Scopus subject areas

  • Health Policy
  • Pharmacology (medical)

Cite this