Neighborhood Cognition Consistent Multi-Agent Reinforcement Learning

  • Hangyu Mao Peking University
  • Wulong Liu Noah's Ark Lab, Huawei
  • Jianye Hao Tianjin University
  • Jun Luo Noah's Ark Lab, Huawei
  • Dong Li Noah's Ark Lab, Huawei
  • Zhengchao Zhang Peking University
  • Jun Wang University College London
  • Zhen Xiao Peking University


Social psychology and real experiences show that cognitive consistency plays an important role to keep human society in order: if people have a more consistent cognition about their environments, they are more likely to achieve better cooperation. Meanwhile, only cognitive consistency within a neighborhood matters because humans only interact directly with their neighbors. Inspired by these observations, we take the first step to introduce neighborhood cognitive consistency (NCC) into multi-agent reinforcement learning (MARL). Our NCC design is quite general and can be easily combined with existing MARL methods. As examples, we propose neighborhood cognition consistent deep Q-learning and Actor-Critic to facilitate large-scale multi-agent cooperations. Extensive experiments on several challenging tasks (i.e., packet routing, wifi configuration and Google football player control) justify the superior performance of our methods compared with state-of-the-art MARL approaches.

AAAI Technical Track: Multiagent Systems