Timezone: »
We propose CaSPR, a method to learn object-centric Canonical Spatiotemporal Point Cloud Representations of dynamically moving or evolving objects. Our goal is to enable information aggregation over time and the interrogation of object state at any spatiotemporal neighborhood in the past, observed or not. Different from previous work, CaSPR learns representations that support spacetime continuity, are robust to variable and irregularly spacetime-sampled point clouds, and generalize to unseen object instances. Our approach divides the problem into two subtasks. First, we explicitly encode time by mapping an input point cloud sequence to a spatiotemporally-canonicalized object space. We then leverage this canonicalization to learn a spatiotemporal latent representation using neural ordinary differential equations and a generative model of dynamically evolving shapes using continuous normalizing flows. We demonstrate the effectiveness of our method on several applications including shape reconstruction, camera pose estimation, continuous spatiotemporal sequence reconstruction, and correspondence estimation from irregularly or intermittently sampled observations.
Author Information
Davis Rempe (Stanford University)
Tolga Birdal (Stanford University)
Yongheng Zhao (University of Padova)
Zan Gojcic (ETH Zürich)
Srinath Sridhar (Brown University)
Leonidas Guibas (stanford.edu)
Related Events (a corresponding poster, oral, or spotlight)
-
2020 Poster: CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations »
Thu. Dec 10th 05:00 -- 07:00 AM Room Poster Session 4 #1300
More from the Same Authors
-
2022 : Breaking the Symmetry: Resolving Symmetry Ambiguities in Equivariant Neural Networks »
Sidhika Balachandar · Adrien Poulenard · Congyue Deng · Leonidas Guibas -
2023 Poster: NeRF Revisited: Fixing Quadrature Instability in Volume Rendering »
Mikaela Angelina Uy · Kiyohiro Nakayama · Guandao Yang · Rahul Thomas · Leonidas Guibas · Ke Li -
2023 Poster: NAP: Neural 3D Articulated Object Prior »
Jiahui Lei · Congyue Deng · William B Shen · Leonidas Guibas · Kostas Daniilidis -
2023 Poster: HyP-NeRF: Learning Improved NeRF Priors using a HyperNetwork »
Bipasha Sen · Gaurav Singh · Aditya Agarwal · Rohith Agaram · Madhava Krishna · Srinath Sridhar -
2023 Poster: Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariance »
Congyue Deng · Jiahui Lei · William B Shen · Kostas Daniilidis · Leonidas Guibas -
2022 Poster: ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model »
Rao Fu · Xiao Zhan · YIWEN CHEN · Daniel Ritchie · Srinath Sridhar -
2022 Poster: NeuForm: Adaptive Overfitting for Neural Shape Editing »
Connor Lin · Niloy Mitra · Gordon Wetzstein · Leonidas Guibas · Paul Guerrero -
2022 Poster: Object Scene Representation Transformer »
Mehdi S. M. Sajjadi · Daniel Duckworth · Aravindh Mahendran · Sjoerd van Steenkiste · Filip Pavetic · Mario Lucic · Leonidas Guibas · Klaus Greff · Thomas Kipf -
2021 Poster: Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks »
Tolga Birdal · Aaron Lou · Leonidas Guibas · Umut Simsekli -
2021 Poster: Leveraging SE(3) Equivariance for Self-supervised Category-Level Object Pose Estimation from Point Clouds »
Xiaolong Li · Yijia Weng · Li Yi · Leonidas Guibas · A. Abbott · Shuran Song · He Wang -
2021 Poster: SketchGen: Generating Constrained CAD Sketches »
Wamiq Para · Shariq Bhat · Paul Guerrero · Tom Kelly · Niloy Mitra · Leonidas Guibas · Peter Wonka -
2020 : QA: Leonidas J. Guibas »
Leonidas Guibas -
2020 : Invited Talk: Leonidas J. Guibas »
Leonidas Guibas -
2020 Poster: Generative 3D Part Assembly via Dynamic Graph Learning »
jialei huang · Guanqi Zhan · Qingnan Fan · Kaichun Mo · Lin Shao · Baoquan Chen · Leonidas Guibas · Hao Dong -
2020 Poster: ShapeFlow: Learnable Deformation Flows Among 3D Shapes »
Chiyu Jiang · Jingwei Huang · Andrea Tagliasacchi · Leonidas Guibas -
2020 Spotlight: ShapeFlow: Learnable Deformation Flows Among 3D Shapes »
Chiyu Jiang · Jingwei Huang · Andrea Tagliasacchi · Leonidas Guibas -
2019 Poster: Multiview Aggregation for Learning Category-Specific Shape Reconstruction »
Srinath Sridhar · Davis Rempe · Julien Valentin · Bouaziz Sofien · Leonidas Guibas -
2019 Poster: A Condition Number for Joint Optimization of Cycle-Consistent Networks »
Leonidas Guibas · Qixing Huang · Zhenxiao Liang -
2019 Spotlight: A Condition Number for Joint Optimization of Cycle-Consistent Networks »
Leonidas Guibas · Qixing Huang · Zhenxiao Liang -
2018 Poster: Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions »
Minhyuk Sung · Hao Su · Ronald Yu · Leonidas Guibas -
2018 Poster: Bayesian Pose Graph Optimization via Bingham Distributions and Tempered Geodesic MCMC »
Tolga Birdal · Umut Simsekli · Mustafa Onur Eken · Slobodan Ilic -
2017 Poster: PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space »
Charles Ruizhongtai Qi · Li Yi · Hao Su · Leonidas Guibas -
2016 Poster: FPNN: Field Probing Neural Networks for 3D Data »
Yangyan Li · Soeren Pirk · Hao Su · Charles R Qi · Leonidas Guibas -
2015 Poster: Deep Knowledge Tracing »
Chris Piech · Jonathan Bassen · Jonathan Huang · Surya Ganguli · Mehran Sahami · Leonidas Guibas · Jascha Sohl-Dickstein -
2013 Poster: Wavelets on Graphs via Deep Learning »
Raif Rustamov · Leonidas Guibas -
2013 Demonstration: Codewebs: a Pedagogical Search Engine for Code Submissions to a MOOC »
Jonathan Huang · Chris Piech · Andy Nguyen · Leonidas Guibas -
2007 Oral: Efficient Inference forDistributions on Permutations »
Jonathan Huang · Carlos Guestrin · Leonidas Guibas -
2007 Poster: Efficient Inference forDistributions on Permutations »
Jonathan Huang · Carlos Guestrin · Leonidas Guibas