Applying and dissecting LSTM neural networks and regularized learning for dynamic inferential modeling

Jicheng LI, S. Joe QIN*

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

2 Citations (Scopus)

Abstract

Deep learning models such as the long short-term memory (LSTM) network have been applied for dynamic inferential modeling. However, many studies apply LSTM as a black-box approach without examining the necessity and usefulness of the internal LSTM gates for inferential modeling. In this paper, we use LSTM as a state space realization and compare it with linear state space modeling and statistical learning methods, including N4SID, partial least squares, the Lasso, and support vector regression. Two case studies on an industrial 660MW boiler and a debutanizer column process indicate that LSTM underperforms all other methods. LSTM is shown to be capable of outperforming linear methods for a simulated reactor process with severely excited nonlinearity in the data. By dissecting the sub-components of a simple LSTM model, the effectiveness of the LSTM gates and nonlinear activation functions is scrutinized.
Original languageEnglish
Article number108264
JournalComputers and Chemical Engineering
Volume175
Early online date26 Apr 2023
DOIs
Publication statusPublished - Jul 2023

Bibliographical note

Funding Information:
Partial financial support for this work is acknowledged from a Natural Science Foundation of China Project ( U20A20189 ), a General Research Fund by the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. 11303421 ), an ITF - Guangdong-Hong Kong Technology Cooperation Funding Scheme (Project Ref. No. GHP/145/20 ), a Math and Application Project ( 2021YFA1003504 ) under the National Key R&D Program, a Shenzhen-Hong Kong-Macau Science and Technology Project Category C ( 9240086 ), an InnoHK initiative of The Government of the HKSAR for the Laboratory for AI-Powered Financial Technologies , and a Collaborative Research Fund (No. C1143-20G ) by RGC of Hong Kong.

Publisher Copyright:
© 2023 Elsevier Ltd

Keywords

  • Dynamic inferential modeling
  • LSTM
  • Partial least squares
  • Regularized learning
  • Subspace identification

Fingerprint

Dive into the research topics of 'Applying and dissecting LSTM neural networks and regularized learning for dynamic inferential modeling'. Together they form a unique fingerprint.

Cite this