Echo state networks with decoupled reservoir states

Bai Zhang, Yue Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Echo state networks (ESNs) are a novel form of recurrent neural networks that provide an efficient and powerful computational model to approximate dynamic nonlinear systems. Why a random, large, fixed recurrent neural network (reservoir) has such astonishing performance in approximating nonlinear systems remains a mystery. In this paper, we first compare two reservoir scenarios in ESNs, i.e. sparsely versus fully connected reservoirs, and show that the eigenvalues of these reservoir weight matrices have the same limit distribution in the complex plane. We discuss the link between the eigenvalues of the reservoir weight matrix and the ESN approximation ability in a simplified ESN case. We propose a new ESN with decoupled reservoir states, in which the neurons in the reservoir are decoupled into single or pairs of neurons. A reservoir state back-elimination strategy is presented, which not only reduces model complexity but also increases numerical stability when calculating the output weights. The proposed model is tested in a communication channel equalization problem and applied to gene expression time series modeling with very promising results.

Original languageEnglish (US)
Title of host publicationProceedings of the 2008 IEEE Workshop on Machine Learning for Signal Processing, MLSP 2008
Pages444-449
Number of pages6
DOIs
StatePublished - Dec 1 2008
Event2008 IEEE Workshop on Machine Learning for Signal Processing, MLSP 2008 - Cancun, Mexico
Duration: Oct 16 2008Oct 19 2008

Publication series

NameProceedings of the 2008 IEEE Workshop on Machine Learning for Signal Processing, MLSP 2008

Other

Other2008 IEEE Workshop on Machine Learning for Signal Processing, MLSP 2008
CountryMexico
CityCancun
Period10/16/0810/19/08

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Echo state networks with decoupled reservoir states'. Together they form a unique fingerprint.

Cite this