Timezone: »

 
Poster
DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenes
Jia-Wei Liu · Yan-Pei Cao · Weijia Mao · Wenqiao Zhang · David Junhao Zhang · Jussi Keppo · Ying Shan · Xiaohu Qie · Mike Zheng Shou

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #423
in Poster Session 4 »

Modeling dynamic scenes is important for many applications such as virtual reality and telepresence. Despite achieving unprecedented fidelity for novel view synthesis in dynamic scenes, existing methods based on Neural Radiance Fields (NeRF) suffer from slow convergence (i.e., model training time measured in days). In this paper, we present DeVRF, a novel representation to accelerate learning dynamic radiance fields. The core of DeVRF is to model both the 3D canonical space and 4D deformation field of a dynamic, non-rigid scene with explicit and discrete voxel-based representations. However, it is quite challenging to train such a representation which has a large number of model parameters, often resulting in overfitting issues. To overcome this challenge, we devise a novel static-to-dynamic learning paradigm together with a new data capture setup that is convenient to deploy in practice. This paradigm unlocks efficient learning of deformable radiance fields via utilizing the 3D volumetric canonical space learnt from multi-view static images to ease the learning of 4D voxel deformation field with only few-view dynamic sequences. To further improve the efficiency of our DeVRF and its synthesized novel view's quality, we conduct thorough explorations and identify a set of strategies. We evaluate DeVRF on both synthetic and real-world dynamic scenes with different types of deformation. Experiments demonstrate that DeVRF achieves two orders of magnitude speedup (100× faster) with on-par high-fidelity results compared to the previous state-of-the-art approaches. The code and dataset are released in https://github.com/showlab/DeVRF.

Keywords: dynamic novel view synthesis. · dynamic radiance field · volumetric representation
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Jia-Wei Liu (National University of Singapore)

I am currently a second-year PhD student in Show Lab, National University of Singapore, advised by Prof. Mike Zheng Shou and Prof. Jussi Keppo. My research topic is 3D avatar and scene.

Yan-Pei Cao (Tsinghua University, Tsinghua University)
Weijia Mao (national university of singaore, National University of Singapore)
Wenqiao Zhang (National University of Singapore)
David Junhao Zhang (National University of Singapore)
Jussi Keppo (National University of Singapore)
Ying Shan (Tencent)
Xiaohu Qie (Tencent)
Mike Zheng Shou (National University of Singapore)

More from the Same Authors