Timezone: »

 
Poster
Hand-Object Interaction Image Generation
Hezhen Hu · Weilun Wang · Wengang Zhou · Houqiang Li

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #636
in Poster Session 4 »

In this work, we are dedicated to a new task, i.e., hand-object interaction image generation, which aims to conditionally generate the hand-object image under the given hand, object and their interaction status. This task is challenging and research-worthy in many potential application scenarios, such as AR/VR games and online shopping, etc. To address this problem, we propose a novel HOGAN framework, which utilizes the expressive model-aware hand-object representation and leverages its inherent topology to build the unified surface space. In this space, we explicitly consider the complex self- and mutual occlusion during interaction. During final image synthesis, we consider different characteristics of hand and object and generate the target image in a split-and-combine manner. For evaluation, we build a comprehensive protocol to access both the fidelity and structure preservation of the generated image. Extensive experiments on two large-scale datasets, i.e., HO3Dv3 and DexYCB, demonstrate the effectiveness and superiority of our framework both quantitatively and qualitatively. The code will be available at https://github.com/play-with-HOI-generation/HOIG.

Keywords: Image Generation · complex occulsion · hand-object interaction
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Hezhen Hu (University of Science and Technology of China)
Weilun Wang (University of Science and Technology of China)
Wengang Zhou (University of Science and Technology of China (USTC))
Houqiang Li (University of Science and Technology of China)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors