TY - JOUR
T1 - Cooperative Data Sensing and Computation Offloading in UAV-assisted Crowdsensing with Multi-agent Deep Reinforcement Learning
AU - CAI, Ting
AU - YANG, Zhihua
AU - CHEN, Yufei
AU - CHEN, Wuhui
AU - ZHENG, Zibin
AU - YU, Yang
AU - DAI, Hong-Ning
N1 - Publisher Copyright:
© 2013 IEEE.
PY - 2022
Y1 - 2022
N2 - Unmanned aerial vehicles (UAVs) can be leveraged in mobile crowdsensing (MCS) to conduct sensing tasks at remote or rural areas through computation offloading and data sensing. Nonetheless, both computation offloading and data sensing have been separately investigated in most existing studies. In this paper, we propose a novel cooperative data sensing and computation offloading scheme for the UAV-assisted MCS system with an aim to maximize the overall system utility. First, a multi-objective function is formulated to evaluate the system utility by jointly considering flight direction, flight distance, task offloading proportion, and server offload selection for each UAV. Then, the problem is modeled as a partially observable Markov decision process and a multi-agent actor-critic algorithm framework is proposed to train the strategy network for UAVs. Due to high delay and energy cost caused by communications among multiple agents, we leverage the critic network to model other agents and to seek equilibrium among all UAVs rather than adopting the explicit channel for information exchange. Furthermore, we introduce attention mechanism to enhance the convergence performance in model training phases. Finally, experimental results demonstrate the effectiveness and applicability of our scheme. Compared with baselines, our algorithm shows significant advantages in convergence performance and system utility.
AB - Unmanned aerial vehicles (UAVs) can be leveraged in mobile crowdsensing (MCS) to conduct sensing tasks at remote or rural areas through computation offloading and data sensing. Nonetheless, both computation offloading and data sensing have been separately investigated in most existing studies. In this paper, we propose a novel cooperative data sensing and computation offloading scheme for the UAV-assisted MCS system with an aim to maximize the overall system utility. First, a multi-objective function is formulated to evaluate the system utility by jointly considering flight direction, flight distance, task offloading proportion, and server offload selection for each UAV. Then, the problem is modeled as a partially observable Markov decision process and a multi-agent actor-critic algorithm framework is proposed to train the strategy network for UAVs. Due to high delay and energy cost caused by communications among multiple agents, we leverage the critic network to model other agents and to seek equilibrium among all UAVs rather than adopting the explicit channel for information exchange. Furthermore, we introduce attention mechanism to enhance the convergence performance in model training phases. Finally, experimental results demonstrate the effectiveness and applicability of our scheme. Compared with baselines, our algorithm shows significant advantages in convergence performance and system utility.
KW - Mobile crowdsensing (MCS)
KW - unmanned aerial vehicle (UAV)
KW - data sensing
KW - computation offloading
KW - deep reinforcement learning (DRL)
UR - http://www.scopus.com/inward/record.url?scp=85138837158&partnerID=8YFLogxK
U2 - 10.1109/TNSE.2021.3121690
DO - 10.1109/TNSE.2021.3121690
M3 - Journal Article (refereed)
AN - SCOPUS:85138837158
SN - 2327-4697
VL - 9
SP - 3197
EP - 3211
JO - IEEE Transactions on Network Science and Engineering
JF - IEEE Transactions on Network Science and Engineering
IS - 5
ER -