A history-derived reward prediction error signal in ventral pallidum

David J. Ottenheimer, Bilal A. Bari, Elissa Sutlief, Kurt M. Fraser, Tabitha H. Kim, Jocelyn M. Richard, Jeremiah Y. Cohen, Patricia H. Janak

Research output: Contribution to journalArticlepeer-review

Abstract

Learning from past interactions with the environment is critical for adaptive behavior. Within 2 the framework of reinforcement learning, the nervous system builds expectations about future 3 reward by computing reward prediction errors (RPEs), the difference between actual and predicted 4 rewards. Correlates of RPEs have been observed in the midbrain dopamine system, which is thought 5 to locally compute this important variable in service of learning. However, the extent to which 6 RPE signals may be computed upstream of the dopamine system is largely unknown. Here, we 7 quantify history-based RPE signals in the ventral pallidum (VP), an input region to the midbrain 8 dopamine system implicated in reward-seeking behavior. We trained rats to associate cues with 9 future delivery of reward and fit computational models to predict individual neuron firing rates 10 at the time of reward delivery. We found that a subset of VP neurons encoded RPEs and did 11 so more robustly than nucleus accumbens, an input to VP. VP RPEs predicted trial-by-trial task 12 engagement, and optogenetic inhibition of VP reduced subsequent task-related reward seeking. 13 Consistent with reinforcement learning, activity of VP RPE cells adapted when rewards were 14 delivered in blocks. We further found that history- and cue-based RPEs were largely separate 15 across the VP neural population. The presence of behaviorally-instructive RPE signals in the VP 16 suggests a pivotal role for this region in value-based computations. 17

Original languageEnglish (US)
JournalUnknown Journal
DOIs
StatePublished - Oct 17 2019

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)
  • Immunology and Microbiology(all)
  • Neuroscience(all)
  • Pharmacology, Toxicology and Pharmaceutics(all)

Fingerprint Dive into the research topics of 'A history-derived reward prediction error signal in ventral pallidum'. Together they form a unique fingerprint.

Cite this