Timezone: »
We present ShapeCrafter, a neural network for recursive text-conditioned 3D shape generation. Existing methods to generate text-conditioned 3D shapes consume an entire text prompt to generate a 3D shape in a single step. However, humans tend to describe shapes recursively---we may start with an initial description and progressively add details based on intermediate results. To capture this recursive process, we introduce a method to generate a 3D shape distribution, conditioned on an initial phrase, that gradually evolves as more phrases are added. Since existing datasets are insufficient for training this approach, we present Text2Shape++, a large dataset of 369K shape--text pairs that supports recursive shape generation. To capture local details that are often used to refine shape descriptions, we build on top of vector-quantized deep implicit functions that generate a distribution of high-quality shapes. Results show that our method can generate shapes consistent with text descriptions, and shapes evolve gradually as more phrases are added. Our method supports shape editing, extrapolation, and can enable new applications in human--machine collaboration for creative design.
Author Information
Rao Fu (Brown University)
Xiao Zhan (Brown University)
YIWEN CHEN (Brown University)
Daniel Ritchie (Brown University)
Srinath Sridhar (Brown University)
More from the Same Authors
-
2021 Poster: HRFormer: High-Resolution Vision Transformer for Dense Predict »
YUHUI YUAN · Rao Fu · Lang Huang · Weihong Lin · Chao Zhang · Xilin Chen · Jingdong Wang -
2020 Poster: CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations »
Davis Rempe · Tolga Birdal · Yongheng Zhao · Zan Gojcic · Srinath Sridhar · Leonidas Guibas -
2020 Spotlight: CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations »
Davis Rempe · Tolga Birdal · Yongheng Zhao · Zan Gojcic · Srinath Sridhar · Leonidas Guibas -
2019 Poster: Multiview Aggregation for Learning Category-Specific Shape Reconstruction »
Srinath Sridhar · Davis Rempe · Julien Valentin · Bouaziz Sofien · Leonidas Guibas -
2018 Poster: Learning to Infer Graphics Programs from Hand-Drawn Images »
Kevin Ellis · Daniel Ritchie · Armando Solar-Lezama · Josh Tenenbaum -
2018 Spotlight: Learning to Infer Graphics Programs from Hand-Drawn Images »
Kevin Ellis · Daniel Ritchie · Armando Solar-Lezama · Josh Tenenbaum