Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features

Daniel R. Mendat; James E. West; Sudarshan Ramenahalli; Ernst Niebur; Andreas G. Andreou

doi:10.1109/CISS.2017.7926180

Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features

Daniel R. Mendat, James E. West, Sudarshan Ramenahalli, Ernst Niebur, Andreas G. Andreou

School of Medicine

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Audio-visual beamforming involves both an acoustic sensor and an omni-camera to form a composite 3D audio-visual representation of the environment. Information from the respective modalities is combined in the process of acoustic localization taking into account high level cognitive features of the signals, namely the presence of specific sounds - speech and tones - which have characteristic signatures in specific spectral bands. We compare the results from two systems. One is a custom-built architecture based on the MH Acoustics Eigenmike microphone array and a consumer grade omni-camera (Bloggie). The other is the commercially available Visisonics Audio-Visual (AV) camera. We show that the performances of the two systems are comparable.

Original language	English (US)
Title of host publication	2017 51st Annual Conference on Information Sciences and Systems, CISS 2017
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781509047802
DOIs	https://doi.org/10.1109/CISS.2017.7926180
State	Published - May 10 2017
Event	51st Annual Conference on Information Sciences and Systems, CISS 2017 - Baltimore, United States Duration: Mar 22 2017 → Mar 24 2017

Publication series

Name	2017 51st Annual Conference on Information Sciences and Systems, CISS 2017

Other

Other	51st Annual Conference on Information Sciences and Systems, CISS 2017
Country/Territory	United States
City	Baltimore
Period	3/22/17 → 3/24/17

ASJC Scopus subject areas

Signal Processing
Information Systems and Management
Computer Networks and Communications
Information Systems

Access to Document

10.1109/CISS.2017.7926180

Cite this

Mendat, D. R., West, J. E., Ramenahalli, S., Niebur, E., & Andreou, A. G. (2017). Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features. In 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017 Article 7926180 (2017 51st Annual Conference on Information Sciences and Systems, CISS 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CISS.2017.7926180

Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features. / Mendat, Daniel R.; West, James E.; Ramenahalli, Sudarshan et al.
2017 51st Annual Conference on Information Sciences and Systems, CISS 2017. Institute of Electrical and Electronics Engineers Inc., 2017. 7926180 (2017 51st Annual Conference on Information Sciences and Systems, CISS 2017).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Mendat, DR, West, JE, Ramenahalli, S, Niebur, E & Andreou, AG 2017, Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features. in 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017., 7926180, 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017, Institute of Electrical and Electronics Engineers Inc., 51st Annual Conference on Information Sciences and Systems, CISS 2017, Baltimore, United States, 3/22/17. https://doi.org/10.1109/CISS.2017.7926180

Mendat DR, West JE, Ramenahalli S, Niebur E, Andreou AG. Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features. In 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017. Institute of Electrical and Electronics Engineers Inc. 2017. 7926180. (2017 51st Annual Conference on Information Sciences and Systems, CISS 2017). doi: 10.1109/CISS.2017.7926180

Mendat, Daniel R. ; West, James E. ; Ramenahalli, Sudarshan et al. / Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features. 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017. Institute of Electrical and Electronics Engineers Inc., 2017. (2017 51st Annual Conference on Information Sciences and Systems, CISS 2017).

@inproceedings{7f461b90acf641f0b604afc79365c4c4,

title = "Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features",

abstract = "Audio-visual beamforming involves both an acoustic sensor and an omni-camera to form a composite 3D audio-visual representation of the environment. Information from the respective modalities is combined in the process of acoustic localization taking into account high level cognitive features of the signals, namely the presence of specific sounds - speech and tones - which have characteristic signatures in specific spectral bands. We compare the results from two systems. One is a custom-built architecture based on the MH Acoustics Eigenmike microphone array and a consumer grade omni-camera (Bloggie). The other is the commercially available Visisonics Audio-Visual (AV) camera. We show that the performances of the two systems are comparable.",

author = "Mendat, {Daniel R.} and West, {James E.} and Sudarshan Ramenahalli and Ernst Niebur and Andreou, {Andreas G.}",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 51st Annual Conference on Information Sciences and Systems, CISS 2017 ; Conference date: 22-03-2017 Through 24-03-2017",

year = "2017",

month = may,

day = "10",

doi = "10.1109/CISS.2017.7926180",

language = "English (US)",

series = "2017 51st Annual Conference on Information Sciences and Systems, CISS 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2017 51st Annual Conference on Information Sciences and Systems, CISS 2017",

}

TY - GEN

T1 - Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features

AU - Mendat, Daniel R.

AU - West, James E.

AU - Ramenahalli, Sudarshan

AU - Niebur, Ernst

AU - Andreou, Andreas G.

PY - 2017/5/10

Y1 - 2017/5/10

N2 - Audio-visual beamforming involves both an acoustic sensor and an omni-camera to form a composite 3D audio-visual representation of the environment. Information from the respective modalities is combined in the process of acoustic localization taking into account high level cognitive features of the signals, namely the presence of specific sounds - speech and tones - which have characteristic signatures in specific spectral bands. We compare the results from two systems. One is a custom-built architecture based on the MH Acoustics Eigenmike microphone array and a consumer grade omni-camera (Bloggie). The other is the commercially available Visisonics Audio-Visual (AV) camera. We show that the performances of the two systems are comparable.

AB - Audio-visual beamforming involves both an acoustic sensor and an omni-camera to form a composite 3D audio-visual representation of the environment. Information from the respective modalities is combined in the process of acoustic localization taking into account high level cognitive features of the signals, namely the presence of specific sounds - speech and tones - which have characteristic signatures in specific spectral bands. We compare the results from two systems. One is a custom-built architecture based on the MH Acoustics Eigenmike microphone array and a consumer grade omni-camera (Bloggie). The other is the commercially available Visisonics Audio-Visual (AV) camera. We show that the performances of the two systems are comparable.

UR - http://www.scopus.com/inward/record.url?scp=85020223397&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020223397&partnerID=8YFLogxK

U2 - 10.1109/CISS.2017.7926180

DO - 10.1109/CISS.2017.7926180

M3 - Conference contribution

AN - SCOPUS:85020223397

T3 - 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017

BT - 2017 51st Annual Conference on Information Sciences and Systems, CISS 2017

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 51st Annual Conference on Information Sciences and Systems, CISS 2017

Y2 - 22 March 2017 through 24 March 2017

ER -

Audio-Visual beamforming with the Eigenmike microphone array an omni-camera and cognitive auditory features

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this