Increasing motor noise impairs reinforcement learning in healthy individuals

Amanda S. Therrien, Daniel M. Wolpert, Amy J Bastian

Research output: Contribution to journalArticle

Abstract

Motor variability from exploration is crucial for reinforcement learning as it allows the nervous system to find new task solutions. However, motor variability from noise can be detrimental to learning and may underlie slowed reinforcement learning performance observed in individuals with cerebellar damage. Here we examine whether artificially increasing noise in healthy individuals slows reinforcement learning in a manner similar to that seen in patients with cerebellar damage. Participants used binary reinforcement to learn to rotate their reach angle in a series of directions. By comparing task performance between conditions with different levels of added noise, we show that adding a high level of noise—matched to a group of patients with cerebellar damage—slows learning. In additional experiments, we show that the detrimental effect of noise may lie in reinforcing incorrect behavior, rather than not reinforcing correct behavior. By comparing performance between healthy participants with added noise and a group of patients with cerebellar damage, we found that added noise does not slow the learning of the control group to the same degree observed in the patient group. Using a mechanistic model, we show that added noise in the present study matched patients’ motor noise and total learning. However, increased exploration in the control group relative to the group with cerebellar damage supports faster learning. Our results suggest that motor noise slows reinforcement learning by impairing the mapping of reward to the correct action and that this may underlie deficits induced by cerebellar damage.

Original languageEnglish (US)
Article numbere0050-18
JournaleNeuro
Volume5
Issue number3
DOIs
StatePublished - May 1 2018

Fingerprint

Noise
Learning
Reinforcement (Psychology)
Control Groups
Task Performance and Analysis
Reward
Nervous System
Healthy Volunteers

Keywords

  • Cerebellum
  • Motor learning
  • Motor noise
  • Reinforcement learning
  • Variability

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Increasing motor noise impairs reinforcement learning in healthy individuals. / Therrien, Amanda S.; Wolpert, Daniel M.; Bastian, Amy J.

In: eNeuro, Vol. 5, No. 3, e0050-18, 01.05.2018.

Research output: Contribution to journalArticle

Therrien, Amanda S. ; Wolpert, Daniel M. ; Bastian, Amy J. / Increasing motor noise impairs reinforcement learning in healthy individuals. In: eNeuro. 2018 ; Vol. 5, No. 3.
@article{224da05029a543a0a5b6248a52f561bf,
title = "Increasing motor noise impairs reinforcement learning in healthy individuals",
abstract = "Motor variability from exploration is crucial for reinforcement learning as it allows the nervous system to find new task solutions. However, motor variability from noise can be detrimental to learning and may underlie slowed reinforcement learning performance observed in individuals with cerebellar damage. Here we examine whether artificially increasing noise in healthy individuals slows reinforcement learning in a manner similar to that seen in patients with cerebellar damage. Participants used binary reinforcement to learn to rotate their reach angle in a series of directions. By comparing task performance between conditions with different levels of added noise, we show that adding a high level of noise—matched to a group of patients with cerebellar damage—slows learning. In additional experiments, we show that the detrimental effect of noise may lie in reinforcing incorrect behavior, rather than not reinforcing correct behavior. By comparing performance between healthy participants with added noise and a group of patients with cerebellar damage, we found that added noise does not slow the learning of the control group to the same degree observed in the patient group. Using a mechanistic model, we show that added noise in the present study matched patients’ motor noise and total learning. However, increased exploration in the control group relative to the group with cerebellar damage supports faster learning. Our results suggest that motor noise slows reinforcement learning by impairing the mapping of reward to the correct action and that this may underlie deficits induced by cerebellar damage.",
keywords = "Cerebellum, Motor learning, Motor noise, Reinforcement learning, Variability",
author = "Therrien, {Amanda S.} and Wolpert, {Daniel M.} and Bastian, {Amy J}",
year = "2018",
month = "5",
day = "1",
doi = "10.1523/ENEURO.0050-18.2018",
language = "English (US)",
volume = "5",
journal = "eNeuro",
issn = "2373-2822",
publisher = "Society for Neuroscience",
number = "3",

}

TY - JOUR

T1 - Increasing motor noise impairs reinforcement learning in healthy individuals

AU - Therrien, Amanda S.

AU - Wolpert, Daniel M.

AU - Bastian, Amy J

PY - 2018/5/1

Y1 - 2018/5/1

N2 - Motor variability from exploration is crucial for reinforcement learning as it allows the nervous system to find new task solutions. However, motor variability from noise can be detrimental to learning and may underlie slowed reinforcement learning performance observed in individuals with cerebellar damage. Here we examine whether artificially increasing noise in healthy individuals slows reinforcement learning in a manner similar to that seen in patients with cerebellar damage. Participants used binary reinforcement to learn to rotate their reach angle in a series of directions. By comparing task performance between conditions with different levels of added noise, we show that adding a high level of noise—matched to a group of patients with cerebellar damage—slows learning. In additional experiments, we show that the detrimental effect of noise may lie in reinforcing incorrect behavior, rather than not reinforcing correct behavior. By comparing performance between healthy participants with added noise and a group of patients with cerebellar damage, we found that added noise does not slow the learning of the control group to the same degree observed in the patient group. Using a mechanistic model, we show that added noise in the present study matched patients’ motor noise and total learning. However, increased exploration in the control group relative to the group with cerebellar damage supports faster learning. Our results suggest that motor noise slows reinforcement learning by impairing the mapping of reward to the correct action and that this may underlie deficits induced by cerebellar damage.

AB - Motor variability from exploration is crucial for reinforcement learning as it allows the nervous system to find new task solutions. However, motor variability from noise can be detrimental to learning and may underlie slowed reinforcement learning performance observed in individuals with cerebellar damage. Here we examine whether artificially increasing noise in healthy individuals slows reinforcement learning in a manner similar to that seen in patients with cerebellar damage. Participants used binary reinforcement to learn to rotate their reach angle in a series of directions. By comparing task performance between conditions with different levels of added noise, we show that adding a high level of noise—matched to a group of patients with cerebellar damage—slows learning. In additional experiments, we show that the detrimental effect of noise may lie in reinforcing incorrect behavior, rather than not reinforcing correct behavior. By comparing performance between healthy participants with added noise and a group of patients with cerebellar damage, we found that added noise does not slow the learning of the control group to the same degree observed in the patient group. Using a mechanistic model, we show that added noise in the present study matched patients’ motor noise and total learning. However, increased exploration in the control group relative to the group with cerebellar damage supports faster learning. Our results suggest that motor noise slows reinforcement learning by impairing the mapping of reward to the correct action and that this may underlie deficits induced by cerebellar damage.

KW - Cerebellum

KW - Motor learning

KW - Motor noise

KW - Reinforcement learning

KW - Variability

UR - http://www.scopus.com/inward/record.url?scp=85052105867&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052105867&partnerID=8YFLogxK

U2 - 10.1523/ENEURO.0050-18.2018

DO - 10.1523/ENEURO.0050-18.2018

M3 - Article

VL - 5

JO - eNeuro

JF - eNeuro

SN - 2373-2822

IS - 3

M1 - e0050-18

ER -