Unsupervised learning via maximizing mutual information in neural population coding

Wentao Huang; Kai Liu; Kechen Zhang

Unsupervised learning via maximizing mutual information in neural population coding

Wentao Huang, Kai Liu, Kechen Zhang

School of Medicine

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Shannon's information theory is a widely used tool for many applications including statistical inference, natural language processing, thermal physics, pattern recognition and neuroscience etc. However, it is a big challenge to effectively calculate the mutual information (MI) for these applications. In this paper, we propose an effective asymptotic bounds and approximation for evaluating MI in the context of neural population coding, especially for the case of high-dimensional inputs. Based on that, an unsupervised framework is presented to learn representations, e.g, complete, overcomplete or un-dercomplete bases from the input datasets. Our experiments on MNIST digits image and natural image patches data sets showed robustness and efficiency for extracting salient features compared to existing methods.

Original language	English (US)
Title of host publication	SS-17-01
Subtitle of host publication	Artificial Intelligene for the Social Good; SS-17-02: Computational Construction Grammar and Natural Language Understanding; SS-17-03: Computational Context: Why It's Important, What It Means, and Can It Be Computed?; SS-17-04: Designing the User Experience of Machine Learning Systems; SS-17-05: Interactive Multisensory Object Perception for Embodied Agents; SS-17-06: Learning from Observation of Humans; SS-17-07: Science of Intelligence: Computational Principles of Natural and Artificial Intelligence; SS-17-08: Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing
Publisher	AI Access Foundation
Pages	575-579
Number of pages	5
Volume	SS-17-01 - SS-17-08
ISBN (Electronic)	9781577357797
State	Published - Jan 1 2017
Event	2017 AAAI Spring Symposium - Stanford, United States Duration: Mar 27 2017 → Mar 29 2017

Other

Other	2017 AAAI Spring Symposium
Country/Territory	United States
City	Stanford
Period	3/27/17 → 3/29/17

ASJC Scopus subject areas

Artificial Intelligence

Cite this

Huang, W., Liu, K., & Zhang, K. (2017). Unsupervised learning via maximizing mutual information in neural population coding. In SS-17-01: Artificial Intelligene for the Social Good; SS-17-02: Computational Construction Grammar and Natural Language Understanding; SS-17-03: Computational Context: Why It's Important, What It Means, and Can It Be Computed?; SS-17-04: Designing the User Experience of Machine Learning Systems; SS-17-05: Interactive Multisensory Object Perception for Embodied Agents; SS-17-06: Learning from Observation of Humans; SS-17-07: Science of Intelligence: Computational Principles of Natural and Artificial Intelligence; SS-17-08: Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing (Vol. SS-17-01 - SS-17-08, pp. 575-579). AI Access Foundation.

Unsupervised learning via maximizing mutual information in neural population coding. / Huang, Wentao; Liu, Kai; Zhang, Kechen.

SS-17-01: Artificial Intelligene for the Social Good; SS-17-02: Computational Construction Grammar and Natural Language Understanding; SS-17-03: Computational Context: Why It's Important, What It Means, and Can It Be Computed?; SS-17-04: Designing the User Experience of Machine Learning Systems; SS-17-05: Interactive Multisensory Object Perception for Embodied Agents; SS-17-06: Learning from Observation of Humans; SS-17-07: Science of Intelligence: Computational Principles of Natural and Artificial Intelligence; SS-17-08: Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing. Vol. SS-17-01 - SS-17-08 AI Access Foundation, 2017. p. 575-579.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Huang, W, Liu, K & Zhang, K 2017, Unsupervised learning via maximizing mutual information in neural population coding. in SS-17-01: Artificial Intelligene for the Social Good; SS-17-02: Computational Construction Grammar and Natural Language Understanding; SS-17-03: Computational Context: Why It's Important, What It Means, and Can It Be Computed?; SS-17-04: Designing the User Experience of Machine Learning Systems; SS-17-05: Interactive Multisensory Object Perception for Embodied Agents; SS-17-06: Learning from Observation of Humans; SS-17-07: Science of Intelligence: Computational Principles of Natural and Artificial Intelligence; SS-17-08: Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing. vol. SS-17-01 - SS-17-08, AI Access Foundation, pp. 575-579, 2017 AAAI Spring Symposium, Stanford, United States, 3/27/17.

Huang W, Liu K, Zhang K. Unsupervised learning via maximizing mutual information in neural population coding. In SS-17-01: Artificial Intelligene for the Social Good; SS-17-02: Computational Construction Grammar and Natural Language Understanding; SS-17-03: Computational Context: Why It's Important, What It Means, and Can It Be Computed?; SS-17-04: Designing the User Experience of Machine Learning Systems; SS-17-05: Interactive Multisensory Object Perception for Embodied Agents; SS-17-06: Learning from Observation of Humans; SS-17-07: Science of Intelligence: Computational Principles of Natural and Artificial Intelligence; SS-17-08: Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing. Vol. SS-17-01 - SS-17-08. AI Access Foundation. 2017. p. 575-579

Huang, Wentao ; Liu, Kai ; Zhang, Kechen. / Unsupervised learning via maximizing mutual information in neural population coding. SS-17-01: Artificial Intelligene for the Social Good; SS-17-02: Computational Construction Grammar and Natural Language Understanding; SS-17-03: Computational Context: Why It's Important, What It Means, and Can It Be Computed?; SS-17-04: Designing the User Experience of Machine Learning Systems; SS-17-05: Interactive Multisensory Object Perception for Embodied Agents; SS-17-06: Learning from Observation of Humans; SS-17-07: Science of Intelligence: Computational Principles of Natural and Artificial Intelligence; SS-17-08: Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing. Vol. SS-17-01 - SS-17-08 AI Access Foundation, 2017. pp. 575-579

@inproceedings{c84e459b346c4c9a8754536842494407,

title = "Unsupervised learning via maximizing mutual information in neural population coding",

abstract = "Shannon's information theory is a widely used tool for many applications including statistical inference, natural language processing, thermal physics, pattern recognition and neuroscience etc. However, it is a big challenge to effectively calculate the mutual information (MI) for these applications. In this paper, we propose an effective asymptotic bounds and approximation for evaluating MI in the context of neural population coding, especially for the case of high-dimensional inputs. Based on that, an unsupervised framework is presented to learn representations, e.g, complete, overcomplete or un-dercomplete bases from the input datasets. Our experiments on MNIST digits image and natural image patches data sets showed robustness and efficiency for extracting salient features compared to existing methods.",

author = "Wentao Huang and Kai Liu and Kechen Zhang",

year = "2017",

month = jan,

day = "1",

language = "English (US)",

volume = "SS-17-01 - SS-17-08",

pages = "575--579",

booktitle = "SS-17-01",

publisher = "AI Access Foundation",

note = "2017 AAAI Spring Symposium ; Conference date: 27-03-2017 Through 29-03-2017",

}

TY - GEN

T1 - Unsupervised learning via maximizing mutual information in neural population coding

AU - Huang, Wentao

AU - Liu, Kai

AU - Zhang, Kechen

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Shannon's information theory is a widely used tool for many applications including statistical inference, natural language processing, thermal physics, pattern recognition and neuroscience etc. However, it is a big challenge to effectively calculate the mutual information (MI) for these applications. In this paper, we propose an effective asymptotic bounds and approximation for evaluating MI in the context of neural population coding, especially for the case of high-dimensional inputs. Based on that, an unsupervised framework is presented to learn representations, e.g, complete, overcomplete or un-dercomplete bases from the input datasets. Our experiments on MNIST digits image and natural image patches data sets showed robustness and efficiency for extracting salient features compared to existing methods.

AB - Shannon's information theory is a widely used tool for many applications including statistical inference, natural language processing, thermal physics, pattern recognition and neuroscience etc. However, it is a big challenge to effectively calculate the mutual information (MI) for these applications. In this paper, we propose an effective asymptotic bounds and approximation for evaluating MI in the context of neural population coding, especially for the case of high-dimensional inputs. Based on that, an unsupervised framework is presented to learn representations, e.g, complete, overcomplete or un-dercomplete bases from the input datasets. Our experiments on MNIST digits image and natural image patches data sets showed robustness and efficiency for extracting salient features compared to existing methods.

UR - http://www.scopus.com/inward/record.url?scp=85028704724&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028704724&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85028704724

VL - SS-17-01 - SS-17-08

SP - 575

EP - 579

BT - SS-17-01

PB - AI Access Foundation

T2 - 2017 AAAI Spring Symposium

Y2 - 27 March 2017 through 29 March 2017

ER -

Unsupervised learning via maximizing mutual information in neural population coding

Abstract

Other

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this