Timezone: »

 
Poster
Action-guided 3D Human Motion Prediction
Jiangxin Sun · Zihang Lin · Xintong Han · Jian-Fang Hu · Jia Xu · Wei-Shi Zheng

Wed Dec 08 12:30 AM -- 02:00 AM (PST) @ Virtual
in Poster Session 3 »

The ability of forecasting future human motion is important for human-machine interaction systems to understand human behaviors and make interaction. In this work, we focus on developing models to predict future human motion from past observed video frames. Motivated by the observation that human motion is closely related to the action being performed, we propose to explore action context to guide motion prediction. Specifically, we construct an action-specific memory bank to store representative motion dynamics for each action category, and design a query-read process to retrieve some motion dynamics from the memory bank. The retrieved dynamics are consistent with the action depicted in the observed video frames and serve as a strong prior knowledge to guide motion prediction. We further formulate an action constraint loss to ensure the global semantic consistency of the predicted motion. Extensive experiments demonstrate the effectiveness of the proposed approach, and we achieve state-of-the-art performance on 3D human motion prediction.

Keywords:

Author Information

Jiangxin Sun (Sun Yat-sen University)
Zihang Lin (SUN YAT-SEN UNIVERSITY)
Xintong Han (Huya Inc)
Jian-Fang Hu (SUN YAT-SEN UNIVERSITY)
Jia Xu (Tencent AI Lab)

I am a principal researcher at Tencent AI Lab. Before returning to China, I was a senior research scientist in the Intel Visual Computing Lab, lead by the awesome Vladlen Koltun. I received my Ph.D. in Computer Sciences at the University of Wisconsin-Madison, with my thesis committee of Prof. Vikas Singh (advisor), Prof. Chuck Dyer, Prof. Jerry Zhu, Prof. Jude Shavlik, and Prof. Mark Craven. I was a visiting student in University of Toronto and in Toyota Technological Institute at Chicago, both working with Prof. Raquel Urtasun. Before graduate school, I obtained my B.S. degree from the Department of Computer Science and Technology at Nanjing University, China. My major interests include computer vision, deep learning, reinforcement learning, and robotics.

Wei-Shi Zheng (SUN YAT-SEN UNIVERSITY)

More from the Same Authors