Multi-Agent Reinforcement Learning with Communication-Constrained Priors

Yang Gao (Nanjing University) · Tianpei Yang (Nanjing University) · Guang Yang (Nanjing University) · Jingwen Qiao (nanjing university) · Yanqing Wu (Southeast University) · Jing Huo (Nanjing University, China) · Xingguo Chen (Nanjing University of Posts and Telecommunications)

benchmarkscommunication impactcommunication-constrained modelcooperative policydistributed decision-makingdual mutual information estimatordynamic environmentsglobal rewardlearning priorlossy communicationmessage distinctionmulti-agent systems reinforcement learning robustness scalability

Communication is one of the effective means to improve the learning of cooperative policy in multi-agent systems. However, in most real-world scenarios, lossy communication is a prevalent issue. Existing multi-agent reinforcement learning with communication, due to their limited scalability and robustness, struggles to apply to complex and dynamic real-world environments. To address these challenges, we propose a generalized communication-constrained model to uniformly characterize communication conditions across different scenarios. Based on this, we utilize it as a learning prior to distinguish between lossy and lossless messages for specific scenarios. Additionally, we decouple the impact of lossy and lossless messages on distributed decision-making, drawing on a dual mutual information estimatior, and introduce a communication-constrained multi-agent reinforcement learning framework, quantifying the impact of communication messages into the global reward. Finally, we validate the effectiveness of our approach across several communication-constrained benchmarks.