TieComm: Learning a Hierarchical Communication Topology Based on Tie Theory

Ming YANG, Renzhi DONG, Yiming WANG, Furui LIU, Yali DU, Mingliang ZHOU, Leong Hou U*

*Corresponding author for this work

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

Abstract

Communication plays an important role in Internet of Things that assists cooperation between devices for better resource management. This work considers the problem of learning cooperative policies using communications in Multi-Agent Reinforcement Learning (MARL), which plays an important role to stabilize agent training and improve the policy learned by enabling the agent to capture more information in partially observable environments. Existing studies either adopt a prior topology by experts or learn a communication topology through a costly process. In this work, we optimize the communication mechanism by exploiting both local agent communications and distant agent communications. Our solution is motivated by tie theory in social networks, where strong ties (close friends) communicate differently with weak ties (distant friends). The proposed novel multi-agent reinforcement learning framework named TieComm, learns a dynamic communication topology consisting of inter- and intra-group communication for efficient policy learning. We factorize the joint multi-agent policy into a centralized tie reasoning policy and decentralized conditional action policies of agents, based on which we propose an alternative updating schema to achieve efficient optimization. Experimental results on Level-Based Foraging and Blind-particle Spread demonstrate the effectiveness of our tie theory based RL framework.

Original languageEnglish
Title of host publicationDatabase Systems for Advanced Applications : 28th International Conference, DASFAA 2023, Proceedings
EditorsXin WANG, Maria Luisa SAPINO, Wook-Shin HAN, Amr EL ABBADI, Gill DOBBIE, Zhiyong FENG, Yingxiao SHAO, Hongzhi YIN
PublisherSpringer, Cham
Pages604-613
Number of pages10
ISBN (Print)9783031306365
DOIs
Publication statusPublished - 2023
Externally publishedYes
Event28th International Conference on Database Systems for Advanced Applications, DASFAA 2023 - Tianjin, China
Duration: 17 Apr 202320 Apr 2023

Publication series

NameLecture Notes in Computer Science
Volume13943
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference28th International Conference on Database Systems for Advanced Applications, DASFAA 2023
Country/TerritoryChina
CityTianjin
Period17/04/2320/04/23

Bibliographical note

Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Keywords

  • Communication Topology
  • Cooperation
  • Multi-agent System
  • Reinforcement Learning
  • Social Welfare

Fingerprint

Dive into the research topics of 'TieComm: Learning a Hierarchical Communication Topology Based on Tie Theory'. Together they form a unique fingerprint.

Cite this