Skip to yearly menu bar Skip to main content


Poster

CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognition

Yuhang Wen · Mengyuan Liu · Songtao Wu · Beichen Ding

[ ] [ Project Page ]
Thu 12 Dec 11 a.m. PST — 2 p.m. PST

Abstract:

Skeleton-based multi-entity action recognition is a challenging task aiming to identify interactive actions or group activities involving multiple diverse entities. Existing single-entity action recognition models often fall short in this task due to the inherent distribution discrepancies among the entity skeletons, leading to suboptimal backbone optimization. To this end, we introduce a Convex Hull Adaptive Shift based multi-Entity action recognition method (CHASE), which mitigates inter-entity distribution gaps and benefits backbone learning. Specifically, we formulate the Implicit Convex Hull Constrained Adaptive Shift (ICHAS) to direct the origin towards the region within the skeleton convex hull. Additionally, we parameterize a lightweight Coefficient Learning Block (CLB), facilitating sample-adaptive weight representation. Moreover, we introduce the Mini-batch Pair-wise Maximum Mean Discrepancy (MPMMD) as an auxiliary objective to guide discrepancy minimization. With ICHAS as the constraint, CLB as learnable variables, and MPMMD as the objective, CHASE finds the optimal sample-adaptive shift that minimizes distribution discrepancies. Extensive experiments on six datasets, including NTU Mutual 11/26, H2O, Assembly101, Collective Activity and Volleyball, consistently verify our approach by seamlessly adapting to single-entity backbones and boosting their performance in multi-entity action recognition. Our code is publicly available at https://anonymous.4open.science/r/CHASE-0 .

Live content is unavailable. Log in and register to view live content