### Abstract

A representation-independent mean-field dynamics is presented for batched TD(λ). The task is learning to predict the outcome of an indirectly observed absorbing Markov process. In the case of linear representations, the discrete-time deterministic iteration is an affine map whose fixed point can be expressed in closed form without the assumption of linearly independent observation vectors. Batched linear TD(λ) is proved to converge with probability 1 for all λ. Theory and simulation agree on a random walk example.

Original language | English (US) |
---|---|

Pages (from-to) | 1403-1419 |

Number of pages | 17 |

Journal | Neural Computation |

Volume | 9 |

Issue number | 7 |

State | Published - Oct 1 1997 |

### Fingerprint

### ASJC Scopus subject areas

- Artificial Intelligence
- Control and Systems Engineering
- Neuroscience(all)

### Cite this

**Mean-Field Theory for Batched TD(λ).** / Pineda, Fernando J.

Research output: Contribution to journal › Article

*Neural Computation*, vol. 9, no. 7, pp. 1403-1419.

}

TY - JOUR

T1 - Mean-Field Theory for Batched TD(λ)

AU - Pineda, Fernando J

PY - 1997/10/1

Y1 - 1997/10/1

N2 - A representation-independent mean-field dynamics is presented for batched TD(λ). The task is learning to predict the outcome of an indirectly observed absorbing Markov process. In the case of linear representations, the discrete-time deterministic iteration is an affine map whose fixed point can be expressed in closed form without the assumption of linearly independent observation vectors. Batched linear TD(λ) is proved to converge with probability 1 for all λ. Theory and simulation agree on a random walk example.

AB - A representation-independent mean-field dynamics is presented for batched TD(λ). The task is learning to predict the outcome of an indirectly observed absorbing Markov process. In the case of linear representations, the discrete-time deterministic iteration is an affine map whose fixed point can be expressed in closed form without the assumption of linearly independent observation vectors. Batched linear TD(λ) is proved to converge with probability 1 for all λ. Theory and simulation agree on a random walk example.

UR - http://www.scopus.com/inward/record.url?scp=0003276733&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0003276733&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:0003276733

VL - 9

SP - 1403

EP - 1419

JO - Neural Computation

JF - Neural Computation

SN - 0899-7667

IS - 7

ER -