Low-Dimensional Vector Autoregressive Modeling for Dynamics Prediction with Maximum Likelihood Estimation

While industrial internet of things and big data have great potential of improving predictions and decision-making, challenges are met in analyzing high dimensional data with complex dynamics and uncertainties. With operational data and rich sensor measurements, the high dimensional time series usually do not have full-dimensional dynamics, which calls for reduced-dimensional analytics. For example, high collinearity usually exists in most routine operation time-series data. Therefore, it is crucial to develop parsimonious modeling to extract reduced-dimensional dynamics.

In reduced dimensional dynamic data modeling, principal component analysis, canonical correlation analysis, and partial least squares have been extended to include dynamics in the reduced dimensional latent space. The work proposed dynamic PCA (DPCA) to extract the auto-correlations in time series via an augmented data matrix. To maximize the predictability in dynamic latent variables (DLV), Dong and Qin proposed dynamic-inner PCA (DiPCA) and dynamic-inner CCA (DiCCA) algorithms. Furthermore, Qin developed a latent vector autoregressive modeling algorithm with a CCA objective (LaVAR-CCA). A latent state space (LaSS-CCA) model is subsequently developed by extending the LaVAR model.

So far, most of the developed methods are formulated and solved in the least squares sense, which does not produce uncertainty and covariance estimates for the modeling errors and estimated parameters. In this work, we propose a probabilistic model to partition the measurement space into a signal subspace admitting the low dimensional DLV dynamics and a static noise subspace, which do not need to be orthogonal to each other. Specifically, we use two expectation-maximization (EM) steps to estimate the DLV dynamics and the signal subspace. Moreover, we apply a statistical constraint to characterize the relationship between the estimated signal and static noise subspaces and use it to estimate the oblique projection. The contributions in this work are as follows.

1) We develop a probabilistic reduced-dimensional vector regressive (PredVAR) model with oblique projections. Our study of using VAR to capture latent dynamics Preprint submitted to AIChE Annual Meeting April 3, 2023 should initiate exploring more dynamic models.
2) We develop an iterative algorithm to update the DLV dynamics and oblique projection estimations alternately. It uniquely uses a statistic constraint together with an EM procedure to select the oblique projection given DLV dynamics.
3) We conduct a simulated case study to demonstrate the strength of our approach compared to two benchmarks, a one-shot algorithm that first identifies the oblique projection and then estimates the DLV dynamics and a counterpart that focuses on the orthogonal projection.