Skip to yearly menu bar Skip to main content

Workshop: NeurIPS 2023 Workshop on Machine Learning for Creativity and Design

DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models

Tsun-Hsuan Johnson Wang · Juntian Zheng · Pingchuan Ma · Yilun Du · Byungchul Kim · Andrew Spielberg · Josh Tenenbaum · Chuang Gan · Daniela Rus

[ ]
[ Poster
Sat 16 Dec 1:30 p.m. PST — 2:30 p.m. PST


This paper presents DiffuseBot , a first step toward efficient automatic robotic andvirtual creature content creation. We propose using physical simulation to guide the generative process of pretrained large-scale 3D diffusion models. Diffusion models pretrained for 3D shapes provide an expressive base distribution that can effectively propose reasonable candidate geometries for soft robots. In order to sample robots in a physics-aware and performance-driven manner, we first optimize the embeddings that condition the diffusion model, skewing the sampling distribution toward better-performing robots as evaluated by our simulator. Then, we reformulate the sampling process that incorporates co-optimization over structure and control.

Chat is not available.