Cooperative Data Sensing and Computation Offloading in UAV-assisted Crowdsensing with Multi-agent Deep Reinforcement Learning

Ting CAI, Zhihua YANG, Yufei CHEN, Wuhui CHEN, Zibin ZHENG, Yang YU, Hong-Ning DAI

Research output: Journal PublicationsJournal Article (refereed)peer-review

55 Citations (Scopus)

Abstract

Unmanned aerial vehicles (UAVs) can be leveraged in mobile crowdsensing (MCS) to conduct sensing tasks at remote or rural areas through computation offloading and data sensing. Nonetheless, both computation offloading and data sensing have been separately investigated in most existing studies. In this paper, we propose a novel cooperative data sensing and computation offloading scheme for the UAV-assisted MCS system with an aim to maximize the overall system utility. First, a multi-objective function is formulated to evaluate the system utility by jointly considering flight direction, flight distance, task offloading proportion, and server offload selection for each UAV. Then, the problem is modeled as a partially observable Markov decision process and a multi-agent actor-critic algorithm framework is proposed to train the strategy network for UAVs. Due to high delay and energy cost caused by communications among multiple agents, we leverage the critic network to model other agents and to seek equilibrium among all UAVs rather than adopting the explicit channel for information exchange. Furthermore, we introduce attention mechanism to enhance the convergence performance in model training phases. Finally, experimental results demonstrate the effectiveness and applicability of our scheme. Compared with baselines, our algorithm shows significant advantages in convergence performance and system utility.
Original languageEnglish
Pages (from-to)3197-3211
Number of pages15
JournalIEEE Transactions on Network Science and Engineering
Volume9
Issue number5
Early online date21 Oct 2021
DOIs
Publication statusPublished - 2022

Bibliographical note

Publisher Copyright:
© 2013 IEEE.

Funding

The work was supported in part by the Key-Area Research and Development Program of Guangdong Province under Grant 2019B020214006, in part by the National Natural Science Foundation of China under Grants 62032025 and 61802450, in part by the National Key Research and Development Program of China under Grant 2020YFB1707603, in part by NSFC-Guangdong Joint Fund Project under Grant U20A6003, in part by the Program for Guangdong Introducing Innovative and Entrepreneurial Teams under Grant 2017ZT07X355, in part by Pearl River Talent Recruitment Program under Grant 2019QN01X130, in part by the Science and Technology Research Program of Chongqing Municipal Education Commission under Grant KJZD-K201802401, and in part by Macao Science and Technology Development Fundunder Macao Funding Scheme for Key R&D Projects under Grant 0025/2019/AKP.

Keywords

  • Mobile crowdsensing (MCS)
  • unmanned aerial vehicle (UAV)
  • data sensing
  • computation offloading
  • deep reinforcement learning (DRL)

Fingerprint

Dive into the research topics of 'Cooperative Data Sensing and Computation Offloading in UAV-assisted Crowdsensing with Multi-agent Deep Reinforcement Learning'. Together they form a unique fingerprint.

Cite this