Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR

Andros Tjandra; Sakriani Sakti; Graham Neubig; Tomoki Toda; Mirna Adriani; Satoshi Nakamura

doi:10.1109/ICASSP.2015.7178827

Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR

Andros Tjandra, Sakriani Sakti, Graham Neubig, Tomoki Toda, Mirna Adriani, Satoshi Nakamura

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

19 Citations (Scopus)

Abstract

This paper explores the use of auditory features based on cochleograms; two dimensional speech features derived from gammatone filters within the convolutional neural network (CNN) framework. Furthermore, we also propose various possibilities to combine cochleogram features with log-mel filter banks or spectrogram features. In particular, we combine within low and high levels of CNN framework which we refer to as low-level and high-level feature combination. As comparison, we also construct the similar configuration with deep neural network (DNN). Performance was evaluated in the framework of hybrid neural network - hidden Markov model (NN-HMM) system on TIMIT phoneme sequence recognition task. The results reveal that cochleogram-spectrogram feature combination provides significant advantages. The best accuracy was obtained by high-level combination of two dimensional cochleogram-spectrogram features using CNN, achieved up to 8.2% relative phoneme error rate (PER) reduction from CNN single features or 19.7% relative PER reduction from DNN single features.

Original language	English
Title of host publication	2015 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	4525-4529
Number of pages	5
ISBN (Electronic)	9781467369978
DOIs	https://doi.org/10.1109/ICASSP.2015.7178827
Publication status	Published - 4 Aug 2015
Event	40th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Brisbane, Australia Duration: 19 Apr 2014 → 24 Apr 2014

Publication series

Name	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume	2015-August
ISSN (Print)	1520-6149

Conference

Conference	40th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015
Country/Territory	Australia
City	Brisbane
Period	19/04/14 → 24/04/14

Keywords

DNN and CNN
Deep learning
cochleogram
feature combination

Access to Document

10.1109/ICASSP.2015.7178827

Cite this

Tjandra, A., Sakti, S., Neubig, G., Toda, T., Adriani, M., & Nakamura, S. (2015). Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR. In 2015 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Proceedings (pp. 4525-4529). Article 7178827 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2015-August). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2015.7178827

Tjandra, Andros ; Sakti, Sakriani ; Neubig, Graham et al. / Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR. 2015 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 4525-4529 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).

@inproceedings{d996ceaae5394cab8ec74f8dc2387455,

title = "Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR",

abstract = "This paper explores the use of auditory features based on cochleograms; two dimensional speech features derived from gammatone filters within the convolutional neural network (CNN) framework. Furthermore, we also propose various possibilities to combine cochleogram features with log-mel filter banks or spectrogram features. In particular, we combine within low and high levels of CNN framework which we refer to as low-level and high-level feature combination. As comparison, we also construct the similar configuration with deep neural network (DNN). Performance was evaluated in the framework of hybrid neural network - hidden Markov model (NN-HMM) system on TIMIT phoneme sequence recognition task. The results reveal that cochleogram-spectrogram feature combination provides significant advantages. The best accuracy was obtained by high-level combination of two dimensional cochleogram-spectrogram features using CNN, achieved up to 8.2% relative phoneme error rate (PER) reduction from CNN single features or 19.7% relative PER reduction from DNN single features.",

keywords = "DNN and CNN, Deep learning, cochleogram, feature combination",

author = "Andros Tjandra and Sakriani Sakti and Graham Neubig and Tomoki Toda and Mirna Adriani and Satoshi Nakamura",

note = "Publisher Copyright: {\textcopyright} 2015 IEEE.; 40th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 ; Conference date: 19-04-2014 Through 24-04-2014",

year = "2015",

month = aug,

day = "4",

doi = "10.1109/ICASSP.2015.7178827",

language = "English",

series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "4525--4529",

booktitle = "2015 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Proceedings",

address = "United States",

}

Tjandra, A, Sakti, S, Neubig, G, Toda, T, Adriani, M & Nakamura, S 2015, Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR. in 2015 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Proceedings., 7178827, ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, vol. 2015-August, Institute of Electrical and Electronics Engineers Inc., pp. 4525-4529, 40th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015, Brisbane, Australia, 19/04/14. https://doi.org/10.1109/ICASSP.2015.7178827

Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR. / Tjandra, Andros; Sakti, Sakriani; Neubig, Graham et al.
2015 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2015. p. 4525-4529 7178827 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2015-August).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR

AU - Tjandra, Andros

AU - Sakti, Sakriani

AU - Neubig, Graham

AU - Toda, Tomoki

AU - Adriani, Mirna

AU - Nakamura, Satoshi

PY - 2015/8/4

Y1 - 2015/8/4

N2 - This paper explores the use of auditory features based on cochleograms; two dimensional speech features derived from gammatone filters within the convolutional neural network (CNN) framework. Furthermore, we also propose various possibilities to combine cochleogram features with log-mel filter banks or spectrogram features. In particular, we combine within low and high levels of CNN framework which we refer to as low-level and high-level feature combination. As comparison, we also construct the similar configuration with deep neural network (DNN). Performance was evaluated in the framework of hybrid neural network - hidden Markov model (NN-HMM) system on TIMIT phoneme sequence recognition task. The results reveal that cochleogram-spectrogram feature combination provides significant advantages. The best accuracy was obtained by high-level combination of two dimensional cochleogram-spectrogram features using CNN, achieved up to 8.2% relative phoneme error rate (PER) reduction from CNN single features or 19.7% relative PER reduction from DNN single features.

AB - This paper explores the use of auditory features based on cochleograms; two dimensional speech features derived from gammatone filters within the convolutional neural network (CNN) framework. Furthermore, we also propose various possibilities to combine cochleogram features with log-mel filter banks or spectrogram features. In particular, we combine within low and high levels of CNN framework which we refer to as low-level and high-level feature combination. As comparison, we also construct the similar configuration with deep neural network (DNN). Performance was evaluated in the framework of hybrid neural network - hidden Markov model (NN-HMM) system on TIMIT phoneme sequence recognition task. The results reveal that cochleogram-spectrogram feature combination provides significant advantages. The best accuracy was obtained by high-level combination of two dimensional cochleogram-spectrogram features using CNN, achieved up to 8.2% relative phoneme error rate (PER) reduction from CNN single features or 19.7% relative PER reduction from DNN single features.

KW - DNN and CNN

KW - Deep learning

KW - cochleogram

KW - feature combination

UR - http://www.scopus.com/inward/record.url?scp=84946016334&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2015.7178827

DO - 10.1109/ICASSP.2015.7178827

M3 - Conference contribution

AN - SCOPUS:84946016334

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

SP - 4525

EP - 4529

BT - 2015 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 40th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015

Y2 - 19 April 2014 through 24 April 2014

ER -

Tjandra A, Sakti S, Neubig G, Toda T, Adriani M, Nakamura S. Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR. In 2015 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2015. p. 4525-4529. 7178827. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). doi: 10.1109/ICASSP.2015.7178827

Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this