Learning reward timing in cortex through reward dependent expression of synaptic plasticity

Jeffrey P. Gavornik, Marshall Shuler, Yonatan Loewenstein, Mark F. Bear, Harel Z. Shouval

Research output: Contribution to journalArticle

Abstract

The ability to represent time is an essential component of cognition but its neural basis is unknown. Although extensively studied both behaviorally and electrophysiologically, a general theoretical framework describing the elementary neural mechanisms used by the brain to learn temporal representations is lacking. It is commonly believed that the underlying cellular mechanisms reside in high order cortical regions but recent studies show sustained neural activity in primary sensory cortices that can represent the timing of expected reward. Here, we show that local cortical networks can learn temporal representations through a simple framework predicated on reward dependent expression of synaptic plasticity. We assert that temporal representations are stored in the lateral synaptic connections between neurons and demonstrate that reward-modulated plasticity is sufficient to learn these representations. We implement our model numerically to explain reward-time learning in the primary visual cortex (V1), demonstrate experimental support, and suggest additional experimentally verifiable predictions.

Original languageEnglish (US)
Pages (from-to)6826-6831
Number of pages6
JournalProceedings of the National Academy of Sciences of the United States of America
Volume106
Issue number16
DOIs
StatePublished - Apr 21 2009

Fingerprint

Neuronal Plasticity
Reward
Learning
Aptitude
Visual Cortex
Cognition
Neurons
Brain

Keywords

  • Reinforcment learning
  • Visual cortex

ASJC Scopus subject areas

  • General

Cite this

Learning reward timing in cortex through reward dependent expression of synaptic plasticity. / Gavornik, Jeffrey P.; Shuler, Marshall; Loewenstein, Yonatan; Bear, Mark F.; Shouval, Harel Z.

In: Proceedings of the National Academy of Sciences of the United States of America, Vol. 106, No. 16, 21.04.2009, p. 6826-6831.

Research output: Contribution to journalArticle

Gavornik, Jeffrey P. ; Shuler, Marshall ; Loewenstein, Yonatan ; Bear, Mark F. ; Shouval, Harel Z. / Learning reward timing in cortex through reward dependent expression of synaptic plasticity. In: Proceedings of the National Academy of Sciences of the United States of America. 2009 ; Vol. 106, No. 16. pp. 6826-6831.
@article{9f6056b9ca054b799b65d7b500223bf2,
title = "Learning reward timing in cortex through reward dependent expression of synaptic plasticity",
abstract = "The ability to represent time is an essential component of cognition but its neural basis is unknown. Although extensively studied both behaviorally and electrophysiologically, a general theoretical framework describing the elementary neural mechanisms used by the brain to learn temporal representations is lacking. It is commonly believed that the underlying cellular mechanisms reside in high order cortical regions but recent studies show sustained neural activity in primary sensory cortices that can represent the timing of expected reward. Here, we show that local cortical networks can learn temporal representations through a simple framework predicated on reward dependent expression of synaptic plasticity. We assert that temporal representations are stored in the lateral synaptic connections between neurons and demonstrate that reward-modulated plasticity is sufficient to learn these representations. We implement our model numerically to explain reward-time learning in the primary visual cortex (V1), demonstrate experimental support, and suggest additional experimentally verifiable predictions.",
keywords = "Reinforcment learning, Visual cortex",
author = "Gavornik, {Jeffrey P.} and Marshall Shuler and Yonatan Loewenstein and Bear, {Mark F.} and Shouval, {Harel Z.}",
year = "2009",
month = "4",
day = "21",
doi = "10.1073/pnas.0901835106",
language = "English (US)",
volume = "106",
pages = "6826--6831",
journal = "Proceedings of the National Academy of Sciences of the United States of America",
issn = "0027-8424",
number = "16",

}

TY - JOUR

T1 - Learning reward timing in cortex through reward dependent expression of synaptic plasticity

AU - Gavornik, Jeffrey P.

AU - Shuler, Marshall

AU - Loewenstein, Yonatan

AU - Bear, Mark F.

AU - Shouval, Harel Z.

PY - 2009/4/21

Y1 - 2009/4/21

N2 - The ability to represent time is an essential component of cognition but its neural basis is unknown. Although extensively studied both behaviorally and electrophysiologically, a general theoretical framework describing the elementary neural mechanisms used by the brain to learn temporal representations is lacking. It is commonly believed that the underlying cellular mechanisms reside in high order cortical regions but recent studies show sustained neural activity in primary sensory cortices that can represent the timing of expected reward. Here, we show that local cortical networks can learn temporal representations through a simple framework predicated on reward dependent expression of synaptic plasticity. We assert that temporal representations are stored in the lateral synaptic connections between neurons and demonstrate that reward-modulated plasticity is sufficient to learn these representations. We implement our model numerically to explain reward-time learning in the primary visual cortex (V1), demonstrate experimental support, and suggest additional experimentally verifiable predictions.

AB - The ability to represent time is an essential component of cognition but its neural basis is unknown. Although extensively studied both behaviorally and electrophysiologically, a general theoretical framework describing the elementary neural mechanisms used by the brain to learn temporal representations is lacking. It is commonly believed that the underlying cellular mechanisms reside in high order cortical regions but recent studies show sustained neural activity in primary sensory cortices that can represent the timing of expected reward. Here, we show that local cortical networks can learn temporal representations through a simple framework predicated on reward dependent expression of synaptic plasticity. We assert that temporal representations are stored in the lateral synaptic connections between neurons and demonstrate that reward-modulated plasticity is sufficient to learn these representations. We implement our model numerically to explain reward-time learning in the primary visual cortex (V1), demonstrate experimental support, and suggest additional experimentally verifiable predictions.

KW - Reinforcment learning

KW - Visual cortex

UR - http://www.scopus.com/inward/record.url?scp=66149143188&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=66149143188&partnerID=8YFLogxK

U2 - 10.1073/pnas.0901835106

DO - 10.1073/pnas.0901835106

M3 - Article

C2 - 19346478

AN - SCOPUS:66149143188

VL - 106

SP - 6826

EP - 6831

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 16

ER -