A history-derived reward prediction error signal in ventral pallidum

Learning from past interactions with the environment is critical for adaptive behavior. Within 2 the framework of reinforcement learning, the nervous system builds expectations about future 3 reward by computing reward prediction errors (RPEs), the difference between actual and predicted 4 rewards. Correlates of RPEs have been observed in the midbrain dopamine system, which is thought 5 to locally compute this important variable in service of learning. However, the extent to which 6 RPE signals may be computed upstream of the dopamine system is largely unknown. Here, we 7 quantify history-based RPE signals in the ventral pallidum (VP), an input region to the midbrain 8 dopamine system implicated in reward-seeking behavior. We trained rats to associate cues with 9 future delivery of reward and fit computational models to predict individual neuron firing rates 10 at the time of reward delivery. We found that a subset of VP neurons encoded RPEs and did 11 so more robustly than nucleus accumbens, an input to VP. VP RPEs predicted trial-by-trial task 12 engagement, and optogenetic inhibition of VP reduced subsequent task-related reward seeking. 13 Consistent with reinforcement learning, activity of VP RPE cells adapted when rewards were 14 delivered in blocks. We further found that history- and cue-based RPEs were largely separate 15 across the VP neural population. The presence of behaviorally-instructive RPE signals in the VP 16 suggests a pivotal role for this region in value-based computations. 17