Timezone: »

Learning to Simulate Self-driven Particles System with Coordinated Policy Optimization
Zhenghao Peng · Quanyi Li · Ka Ming Hui · Chunxiao Liu · Bolei Zhou

Wed Dec 08 12:30 AM -- 02:00 AM (PST) @

Self-Driven Particles (SDP) describe a category of multi-agent systems common in everyday life, such as flocking birds and traffic flows. In a SDP system, each agent pursues its own goal and constantly changes its cooperative or competitive behaviors with its nearby agents. Manually designing the controllers for such SDP system is time-consuming, while the resulting emergent behaviors are often not realistic nor generalizable. Thus the realistic simulation of SDP systems remains challenging. Reinforcement learning provides an appealing alternative for automating the development of the controller for SDP. However, previous multi-agent reinforcement learning (MARL) methods define the agents to be teammates or enemies before hand, which fail to capture the essence of SDP where the role of each agent varies to be cooperative or competitive even within one episode. To simulate SDP with MARL, a key challenge is to coordinate agents' behaviors while still maximizing individual objectives. Taking traffic simulation as the testing bed, in this work we develop a novel MARL method called Coordinated Policy Optimization (CoPO), which incorporates social psychology principle to learn neural controller for SDP. Experiments show that the proposed method can achieve superior performance compared to MARL baselines in various metrics. Noticeably the trained vehicles exhibit complex and diverse social behaviors that improve performance and safety of the population as a whole. Demo video and source code are available at: https://decisionforce.github.io/CoPO/

Author Information

Zhenghao Peng (The Chinese University of Hong Kong)
Quanyi Li (cuhk)
Ka Ming Hui (The Chinese University of Hong Kong)
Chunxiao Liu (Sensetime Research)
Bolei Zhou (Massachusetts Institute of Technology)

More from the Same Authors

  • 2022 Poster: Human-AI Shared Control via Policy Dissection »
    Quanyi Li · Zhenghao Peng · Haibin Wu · Lan Feng · Bolei Zhou
  • 2021 Poster: Data-Efficient Instance Generation from Instance Discrimination »
    Ceyuan Yang · Yujun Shen · Yinghao Xu · Bolei Zhou
  • 2018 : Poster Session 1 + Coffee »
    Tom Van de Wiele · Rui Zhao · J. Fernando Hernandez-Garcia · Fabio Pardo · Xian Yeow Lee · Xiaolin Andy Li · Marcin Andrychowicz · Jie Tang · Suraj Nair · Juhyeon Lee · C├ędric Colas · S. M. Ali Eslami · Yen-Chen Wu · Stephen McAleer · Ryan Julian · Yang Xue · Matthia Sabatelli · Pranav Shyam · Alexandros Kalousis · Giovanni Montana · Emanuele Pesce · Felix Leibfried · Zhanpeng He · Chunxiao Liu · Yanjun Li · Yoshihide Sawada · Alexander Pashevich · Tejas Kulkarni · Keiran Paster · Luca Rigazio · Quan Vuong · Hyunggon Park · Minhae Kwon · Rivindu Weerasekera · Shamane Siriwardhanaa · Rui Wang · Ozsel Kilinc · Keith Ross · Yizhou Wang · Simon Schmitt · Thomas Anthony · Evan Cater · Forest Agostinelli · Tegg Sung · Shirou Maruyama · Alexander Shmakov · Devin Schwab · Mohammad Firouzi · Glen Berseth · Denis Osipychev · Jesse Farebrother · Jianlan Luo · William Agnew · Peter Vrancx · Jonathan Heek · Catalin Ionescu · Haiyan Yin · Megumi Miyashita · Nathan Jay · Noga H. Rotman · Sam Leroux · Shaileshh Bojja Venkatakrishnan · Henri Schmidt · Jack Terwilliger · Ishan Durugkar · Jonathan Sauder · David Kas · Arash Tavakoli · Alain-Sam Cohen · Philip Bontrager · Adam Lerer · Thomas Paine · Ahmed Khalifa · Ruben Rodriguez · Avi Singh · Yiming Zhang
  • 2017 : Posters 1 »
    J.P. Lewis · Housam Khalifa Bashier Babiker · Zhongang Qi · Laura Rieger · Ning Xie · Filip Dabek · Koushik Nagasubramanian · Bolei Zhou · Dieuwke Hupkes · CHUN-HAO CHANG · Pamela K Douglas · Enea Ceolini · Derek Doran · Yan Liu · Fuxin Li · Randolph Goebel
  • 2017 : Panel discussion with Hanna Wallach, Kiri Wagstaff, Suchi Saria, Bolei Zhou, and Zack Lipton. Moderated by Rich Caruana. »
    Bolei Zhou
  • 2014 Poster: Learning Deep Features for Scene Recognition using Places Database »
    Bolei Zhou · Agata Lapedriza · Jianxiong Xiao · Antonio Torralba · Aude Oliva
  • 2014 Spotlight: Learning Deep Features for Scene Recognition using Places Database »
    Bolei Zhou · Agata Lapedriza · Jianxiong Xiao · Antonio Torralba · Aude Oliva