Predictive deep reinforcement learning with multi-agent systems for adaptive time series forecasting

Reinforcement learning has been increasingly applied in monitoring applications because of its ability to learn from previous experiences and make adaptive decisions. However, existing machine learning-based health monitoring applications are mostly supervised learning algorithms, trained on labels, and they cannot make adaptive decisions in an uncertain, complex environment. This study proposes a novel and generic system, predictive deep reinforcement learning (PDRL), with multiple RL agents in a time series forecasting environment. The proposed generic framework accommodates virtual Deep Q Network (DQN) agents to monitor predicted future states of a complex environment with a well-defined reward policy so that the agent learns existing knowledge while maximizing their rewards. In the evaluation process of the proposed framework, three DRL agents were deployed to monitor a subject’s future heart rate, respiration, and temperature predicted using a BiLSTM model. With each iteration, the three agents were able to learn the associated patterns, and their cumulative rewards gradually increased. It outperformed the baseline models for all three monitoring agents. The proposed PDRL framework achieves state-of-the-art performance in time series forecasting by effectively integrating reinforcement learning agents with deep learning-based prediction. The proposed DRL agents and deep learning model in the PDRL framework are customized to enable transfer learning in other forecasting applications like traffic and weather, and monitor their states. The PDRL framework is able to learn the future states of the traffic and weather forecasting, and the cumulative rewards are gradually increasing over each episode.