Timezone: »
Poster
Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps
Yue Hu · Shaoheng Fang · Zixing Lei · Yiqi Zhong · Siheng Chen
Multi-agent collaborative perception could significantly upgrade the perception performance by enabling agents to share complementary information with each other through communication. It inevitably results in a fundamental trade-off between perception performance and communication bandwidth. To tackle this bottleneck issue, we propose a spatial confidence map, which reflects the spatial heterogeneity of perceptual information. It empowers agents to only share spatially sparse, yet perceptually critical information, contributing to where to communicate. Based on this novel spatial confidence map, we propose Where2comm, a communication-efficient collaborative perception framework. Where2comm has two distinct advantages: i) it considers pragmatic compression and uses less communication to achieve higher perception performance by focusing on perceptually critical areas; and ii) it can handle varying communication bandwidth by dynamically adjusting spatial areas involved in communication. To evaluate Where2comm, we consider 3D object detection in both real-world and simulation scenarios with two modalities (camera/LiDAR) and two agent types (cars/drones) on four datasets: OPV2V, V2X-Sim, DAIR-V2X, and our original CoPerception-UAVs. Where2comm consistently outperforms previous methods; for example, it achieves more than $100,000 \times$ lower communication volume and still outperforms DiscoNet and V2X-ViT on OPV2V. Our code is available at~\url{https://github.com/MediaBrain-SJTU/where2comm}.
Author Information
Yue Hu (Shanghai Jiao Tong University)
Shaoheng Fang (Shanghai Jiao Tong University)
Zixing Lei (Shanghai Jiaotong University)
Yiqi Zhong (University of Southern California)
Siheng Chen (Shanghai Jiao Tong University)
More from the Same Authors
-
2023 Poster: Emergent Communication in Interactive Sketch Question Answering »
Zixing Lei · Yiming Zhang · Yuxin Xiong · Siheng Chen -
2023 Poster: Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow »
Sizhe Wei · Yuxi Wei · Yue Hu · Yifan Lu · Yiqi Zhong · Siheng Chen · Ya Zhang -
2022 Spotlight: Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps »
Yue Hu · Shaoheng Fang · Zixing Lei · Yiqi Zhong · Siheng Chen -
2021 Poster: Collaborative Uncertainty in Multi-Agent Trajectory Forecasting »
Bohan Tang · Yiqi Zhong · Ulrich Neumann · Gang Wang · Siheng Chen · Ya Zhang -
2020 Poster: Graph Cross Networks with Vertex Infomax Pooling »
Maosen Li · Siheng Chen · Ya Zhang · Ivor Tsang -
2020 Oral: Graph Cross Networks with Vertex Infomax Pooling »
Maosen Li · Siheng Chen · Ya Zhang · Ivor Tsang -
2019 Poster: Deep RGB-D Canonical Correlation Analysis For Sparse Depth Completion »
Yiqi Zhong · Cho-Ying Wu · Suya You · Ulrich Neumann