Timezone: »
We propose a new representation for encoding 3D shapes as neural fields. The representation is designed to be compatible with the transformer architecture and to benefit both shape reconstruction and shape generation. Existing works on neural fields are grid-based representations with latents being defined on a regular grid. In contrast, we define latents on irregular grids which facilitates our representation to be sparse and adaptive. In the context of shape reconstruction from point clouds, our shape representation built on irregular grids improves upon grid-based methods in terms of reconstruction accuracy. For shape generation, our representation promotes high-quality shape generation using auto-regressive probabilistic models. We show different applications that improve over the current state of the art. First, we show results of probabilistic shape reconstruction from a single higher resolution image. Second, we train a probabilistic model conditioned on very low resolution images. Third, we apply our model to category-conditioned generation. All probabilistic experiments confirm that we are able to generate detailed and high quality shapes to yield the new state of the art in generative 3D shape modeling.
Author Information
Biao Zhang (KAUST)
Matthias Niessner (Technical University of Munich)
Peter Wonka (KAUST)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: 3DILG: Irregular Latent Grids for 3D Generative Modeling »
Wed. Nov 30th through Dec 1st Room Hall J #343
More from the Same Authors
-
2023 Poster: DiffComplete: Diffusion-based Generative 3D Shape Completion »
Ruihang Chu · Enze Xie · Shentong Mo · Zhenguo Li · Matthias Niessner · Chi-Wing Fu · Jiaya Jia -
2023 Poster: DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation »
Shentong Mo · Enze Xie · Ruihang Chu · Lanqing Hong · Matthias Niessner · Zhenguo Li -
2023 Poster: SLIBO-Net: Floorplan Reconstruction via Slicing Box Representation with Local Geometry Regularization »
Jheng-Wei Su · Kuei-Yu Tung · Chi-Han Peng · Peter Wonka · Hung-Kuo Chu -
2023 Poster: Estimating Generic 3D Room Structures from 2D Annotations »
Denys Rozumnyi · Stefan Popov · Kevis-kokitsi Maninis · Matthias Niessner · Vittorio Ferrari -
2022 Spotlight: Lightning Talks 6A-3 »
Junyu Xie · Chengliang Zhong · Ali Ayub · Sravanti Addepalli · Harsh Rangwani · Jiapeng Tang · Yuchen Rao · Zhiying Jiang · Yuqi Wang · Xingzhe He · Gene Chou · Ilya Chugunov · Samyak Jain · Yuntao Chen · Weidi Xie · Sumukh K Aithal · Carter Fendley · Lev Markhasin · Yiqin Dai · Peixing You · Bastian Wandt · Yinyu Nie · Helge Rhodin · Felix Heide · Ji Xin · Angela Dai · Andrew Zisserman · Bi Wang · Xiaoxue Chen · Mayank Mishra · ZHAO-XIANG ZHANG · Venkatesh Babu R · Justus Thies · Ming Li · Hao Zhao · Venkatesh Babu R · Jimmy Lin · Fuchun Sun · Matthias Niessner · Guyue Zhou · Xiaodong Mu · Chuang Gan · Wenbing Huang -
2022 Spotlight: Neural Shape Deformation Priors »
Jiapeng Tang · Lev Markhasin · Bi Wang · Justus Thies · Matthias Niessner -
2022 Spotlight: EpiGRAF: Rethinking training of 3D GANs »
Ivan Skorokhodov · Sergey Tulyakov · Yiqun Wang · Peter Wonka -
2022 Spotlight: HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details »
Yiqun Wang · Ivan Skorokhodov · Peter Wonka -
2022 Spotlight: Lightning Talks 5B-1 »
Devansh Arpit · Xiaojun Xu · Zifan Shi · Ivan Skorokhodov · Shayan Shekarforoush · Zhan Tong · Yiqun Wang · Shichong Peng · Linyi Li · Ivan Skorokhodov · Huan Wang · Yibing Song · David Lindell · Yinghao Xu · Seyed Alireza Moazenipourasil · Sergey Tulyakov · Peter Wonka · Yiqun Wang · Ke Li · David Fleet · Yujun Shen · Yingbo Zhou · Bo Li · Jue Wang · Peter Wonka · Marcus Brubaker · Caiming Xiong · Limin Wang · Deli Zhao · Qifeng Chen · Dit-Yan Yeung -
2022 Poster: Neural Shape Deformation Priors »
Jiapeng Tang · Lev Markhasin · Bi Wang · Justus Thies · Matthias Niessner -
2022 Poster: EpiGRAF: Rethinking training of 3D GANs »
Ivan Skorokhodov · Sergey Tulyakov · Yiqun Wang · Peter Wonka -
2022 Poster: HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details »
Yiqun Wang · Ivan Skorokhodov · Peter Wonka -
2022 Poster: The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes »
Peter Kocsis · Peter Súkeník · Guillem Braso · Matthias Niessner · Laura Leal-Taixé · Ismail Elezi -
2021 Poster: TransformerFusion: Monocular RGB Scene Reconstruction using Transformers »
Aljaz Bozic · Pablo Palafox · Justus Thies · Angela Dai · Matthias Niessner -
2021 Poster: Panoptic 3D Scene Reconstruction From a Single RGB Image »
Manuel Dahnert · Ji Hou · Matthias Niessner · Angela Dai -
2020 Poster: Neural Non-Rigid Tracking »
Aljaz Bozic · Pablo Palafox · Michael Zollhöfer · Angela Dai · Justus Thies · Matthias Niessner