Syllable based DNN-HMM Cantonese speech-to-text system

Timothy WONG; Claire W.Y. LI; Sam LAM; Billy CHIU; Qin LU; Minglei LI; Dan XIONG; Roy S. YU; Vincent T. Y. NG

Syllable based DNN-HMM Cantonese speech-to-text system

Timothy WONG, Claire W.Y. LI, Sam LAM, Billy CHIU, Qin LU, Minglei LI, Dan XIONG, Roy S. YU, Vincent T. Y. NG

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Research › peer-review

2 Citations (Scopus)

Abstract

This paper reports our work on building up a Cantonese Speech-to-Text (STT) system with a syllable based acoustic model. This is a part of an effort in building a STT system to aid dyslexic students who have cognitive deficiency in writing skills but have no problem expressing their ideas through speech. For Cantonese speech recognition, the basic unit of acoustic models can either be the conventional Initial-Final (IF) syllables, or the Onset-Nucleus-Coda (ONC) syllables where finals are further split into nucleus and coda to reflect the intra-syllable variations in Cantonese. By using the Kaldi toolkit, our system is trained using the stochastic gradient descent optimization model with the aid of GPUs for the hybrid Deep Neural Network and Hidden Markov Model (DNN-HMM) with and without I-vector based speaker adaptive training technique. The input features of the same Gaussian Mixture Model with speaker adaptive training (GMM-SAT) to DNN are used in all cases. Experiments show that the ONC-based syllable acoustic modeling with I-vector based DNN-HMM achieves the best performance with the word error rate (WER) of 9.66% and the real time factor (RTF) of 1.38812.

Original language	English
Title of host publication	Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016
Editors	Nicoletta CALZOLARI, Khalid CHOUKRI, Helene MAZO, Asuncion MORENO, Thierry DECLERCK, Sara GOGGI, Marko GROBELINK, Jan ODIJK, Stelios PIPERIDIS, Bente MAEGAARD, Joseph MARIANI
Publisher	European Language Resources Association (ELRA)
Pages	3856-3862
Number of pages	7
ISBN (Electronic)	9782951740891
Publication status	Published - 2016
Externally published	Yes
Event	10th International Conference on Language Resources and Evaluation, LREC 2016 - Portoroz, Slovenia Duration: 23 May 2016 → 28 May 2016

Publication series

Name	Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016

Conference

Conference	10th International Conference on Language Resources and Evaluation, LREC 2016
Country/Territory	Slovenia
City	Portoroz
Period	23/05/16 → 28/05/16

Funding

The project is partially supported by the Hong Kong Government’s ITF fund (ITS/072/14) and the Hong Kong Polytechnic University project RTVU. Our special thanks to all Kaldi community.

Keywords

Cantonese speech recognition
DNN-HMM
Kaldi toolkit
Onset-nucleus-coda syllable scheme

Cite this

WONG, T., LI, C. W. Y., LAM, S., CHIU, B., LU, Q., LI, M., XIONG, D., YU, R. S., & NG, V. T. Y. (2016). Syllable based DNN-HMM Cantonese speech-to-text system. In N. CALZOLARI, K. CHOUKRI, H. MAZO, A. MORENO, T. DECLERCK, S. GOGGI, M. GROBELINK, J. ODIJK, S. PIPERIDIS, B. MAEGAARD, & J. MARIANI (Eds.), Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016 (pp. 3856-3862). (Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016). European Language Resources Association (ELRA).

WONG, Timothy ; LI, Claire W.Y. ; LAM, Sam et al. / Syllable based DNN-HMM Cantonese speech-to-text system. Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016. editor / Nicoletta CALZOLARI ; Khalid CHOUKRI ; Helene MAZO ; Asuncion MORENO ; Thierry DECLERCK ; Sara GOGGI ; Marko GROBELINK ; Jan ODIJK ; Stelios PIPERIDIS ; Bente MAEGAARD ; Joseph MARIANI. European Language Resources Association (ELRA), 2016. pp. 3856-3862 (Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016).

@inproceedings{dd6dffb9a4a740498ca4d60517f6c812,

title = "Syllable based DNN-HMM Cantonese speech-to-text system",

abstract = "This paper reports our work on building up a Cantonese Speech-to-Text (STT) system with a syllable based acoustic model. This is a part of an effort in building a STT system to aid dyslexic students who have cognitive deficiency in writing skills but have no problem expressing their ideas through speech. For Cantonese speech recognition, the basic unit of acoustic models can either be the conventional Initial-Final (IF) syllables, or the Onset-Nucleus-Coda (ONC) syllables where finals are further split into nucleus and coda to reflect the intra-syllable variations in Cantonese. By using the Kaldi toolkit, our system is trained using the stochastic gradient descent optimization model with the aid of GPUs for the hybrid Deep Neural Network and Hidden Markov Model (DNN-HMM) with and without I-vector based speaker adaptive training technique. The input features of the same Gaussian Mixture Model with speaker adaptive training (GMM-SAT) to DNN are used in all cases. Experiments show that the ONC-based syllable acoustic modeling with I-vector based DNN-HMM achieves the best performance with the word error rate (WER) of 9.66% and the real time factor (RTF) of 1.38812.",

keywords = "Cantonese speech recognition, DNN-HMM, Kaldi toolkit, Onset-nucleus-coda syllable scheme",

author = "Timothy WONG and LI, {Claire W.Y.} and Sam LAM and Billy CHIU and Qin LU and Minglei LI and Dan XIONG and YU, {Roy S.} and NG, {Vincent T. Y.}",

year = "2016",

language = "English",

series = "Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016",

publisher = "European Language Resources Association (ELRA)",

pages = "3856--3862",

editor = "Nicoletta CALZOLARI and Khalid CHOUKRI and Helene MAZO and Asuncion MORENO and Thierry DECLERCK and Sara GOGGI and Marko GROBELINK and Jan ODIJK and Stelios PIPERIDIS and Bente MAEGAARD and Joseph MARIANI",

booktitle = "Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016",

note = "10th International Conference on Language Resources and Evaluation, LREC 2016 ; Conference date: 23-05-2016 Through 28-05-2016",

}

WONG, T, LI, CWY, LAM, S, CHIU, B, LU, Q, LI, M, XIONG, D, YU, RS & NG, VTY 2016, Syllable based DNN-HMM Cantonese speech-to-text system. in N CALZOLARI, K CHOUKRI, H MAZO, A MORENO, T DECLERCK, S GOGGI, M GROBELINK, J ODIJK, S PIPERIDIS, B MAEGAARD & J MARIANI (eds), Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016. Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016, European Language Resources Association (ELRA), pp. 3856-3862, 10th International Conference on Language Resources and Evaluation, LREC 2016, Portoroz, Slovenia, 23/05/16.

Syllable based DNN-HMM Cantonese speech-to-text system. / WONG, Timothy; LI, Claire W.Y.; LAM, Sam et al.
Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016. ed. / Nicoletta CALZOLARI; Khalid CHOUKRI; Helene MAZO; Asuncion MORENO; Thierry DECLERCK; Sara GOGGI; Marko GROBELINK; Jan ODIJK; Stelios PIPERIDIS; Bente MAEGAARD; Joseph MARIANI. European Language Resources Association (ELRA), 2016. p. 3856-3862 (Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016).

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Research › peer-review

TY - GEN

T1 - Syllable based DNN-HMM Cantonese speech-to-text system

AU - WONG, Timothy

AU - LI, Claire W.Y.

AU - LAM, Sam

AU - CHIU, Billy

AU - LU, Qin

AU - LI, Minglei

AU - XIONG, Dan

AU - YU, Roy S.

AU - NG, Vincent T. Y.

PY - 2016

Y1 - 2016

N2 - This paper reports our work on building up a Cantonese Speech-to-Text (STT) system with a syllable based acoustic model. This is a part of an effort in building a STT system to aid dyslexic students who have cognitive deficiency in writing skills but have no problem expressing their ideas through speech. For Cantonese speech recognition, the basic unit of acoustic models can either be the conventional Initial-Final (IF) syllables, or the Onset-Nucleus-Coda (ONC) syllables where finals are further split into nucleus and coda to reflect the intra-syllable variations in Cantonese. By using the Kaldi toolkit, our system is trained using the stochastic gradient descent optimization model with the aid of GPUs for the hybrid Deep Neural Network and Hidden Markov Model (DNN-HMM) with and without I-vector based speaker adaptive training technique. The input features of the same Gaussian Mixture Model with speaker adaptive training (GMM-SAT) to DNN are used in all cases. Experiments show that the ONC-based syllable acoustic modeling with I-vector based DNN-HMM achieves the best performance with the word error rate (WER) of 9.66% and the real time factor (RTF) of 1.38812.

AB - This paper reports our work on building up a Cantonese Speech-to-Text (STT) system with a syllable based acoustic model. This is a part of an effort in building a STT system to aid dyslexic students who have cognitive deficiency in writing skills but have no problem expressing their ideas through speech. For Cantonese speech recognition, the basic unit of acoustic models can either be the conventional Initial-Final (IF) syllables, or the Onset-Nucleus-Coda (ONC) syllables where finals are further split into nucleus and coda to reflect the intra-syllable variations in Cantonese. By using the Kaldi toolkit, our system is trained using the stochastic gradient descent optimization model with the aid of GPUs for the hybrid Deep Neural Network and Hidden Markov Model (DNN-HMM) with and without I-vector based speaker adaptive training technique. The input features of the same Gaussian Mixture Model with speaker adaptive training (GMM-SAT) to DNN are used in all cases. Experiments show that the ONC-based syllable acoustic modeling with I-vector based DNN-HMM achieves the best performance with the word error rate (WER) of 9.66% and the real time factor (RTF) of 1.38812.

KW - Cantonese speech recognition

KW - DNN-HMM

KW - Kaldi toolkit

KW - Onset-nucleus-coda syllable scheme

UR - http://www.scopus.com/inward/record.url?scp=85037150155&partnerID=8YFLogxK

M3 - Conference paper (refereed)

AN - SCOPUS:85037150155

T3 - Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016

SP - 3856

EP - 3862

BT - Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016

A2 - CALZOLARI, Nicoletta

A2 - CHOUKRI, Khalid

A2 - MAZO, Helene

A2 - MORENO, Asuncion

A2 - DECLERCK, Thierry

A2 - GOGGI, Sara

A2 - GROBELINK, Marko

A2 - ODIJK, Jan

A2 - PIPERIDIS, Stelios

A2 - MAEGAARD, Bente

A2 - MARIANI, Joseph

PB - European Language Resources Association (ELRA)

T2 - 10th International Conference on Language Resources and Evaluation, LREC 2016

Y2 - 23 May 2016 through 28 May 2016

ER -

WONG T, LI CWY, LAM S, CHIU B, LU Q, LI M et al. Syllable based DNN-HMM Cantonese speech-to-text system. In CALZOLARI N, CHOUKRI K, MAZO H, MORENO A, DECLERCK T, GOGGI S, GROBELINK M, ODIJK J, PIPERIDIS S, MAEGAARD B, MARIANI J, editors, Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016. European Language Resources Association (ELRA). 2016. p. 3856-3862. (Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016).

Syllable based DNN-HMM Cantonese speech-to-text system

Abstract

Publication series

Conference

Funding

Keywords

Other files and links

Fingerprint

Cite this