A comparison of expert and nonexpert raters using the consensual assessment technique

James C. Kaufman, John Baer, Jason C. Cole, Janel D. Sexton

Research output: Contribution to journalArticlepeer-review

Abstract

The Consensual Assessment Technique (CAT) is one of the most highly regarded assessment tools in creativity, but it is often difficult and/or expensive to assemble the teams of experts required by the CAT. Some researchers have tried using nonexpert raters in their place, but the validity of replacing experts with nonexperts has not been adequately tested. Expert (n = 10) and nonexpert (n = 106) creativity ratings of 205 poems were compared and found to be quite different, making the simple replacement of experts by nonexpert raters suspect. Nonexpert raters' judgments of creativity were inconsistent (showing low interrater reliability) and did not match those of the expert raters. Implications are discussed, including the appropriate selection of expert raters for different kinds of creativity assessment.

Original languageEnglish (US)
Pages (from-to)171-178
Number of pages8
JournalCreativity Research Journal
Volume20
Issue number2
DOIs
StatePublished - Apr 1 2008

ASJC Scopus subject areas

  • Visual Arts and Performing Arts
  • Developmental and Educational Psychology
  • Psychology (miscellaneous)

Fingerprint Dive into the research topics of 'A comparison of expert and nonexpert raters using the consensual assessment technique'. Together they form a unique fingerprint.

Cite this