Timezone: »

 
Poster
MarioNette: Self-Supervised Sprite Learning
Dmitriy Smirnov · MICHAEL GHARBI · Matthew Fisher · Vitor Guizilini · Alexei Efros · Justin Solomon

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @
in Poster Session 1 »

Artists and video game designers often construct 2D animations using libraries of sprites---textured patches of objects and characters. We propose a deep learning approach that decomposes sprite-based video animations into a disentangled representation of recurring graphic elements in a self-supervised manner. By jointly learning a dictionary of possibly transparent patches and training a network that places them onto a canvas, we deconstruct sprite-based content into a sparse, consistent, and explicit representation that can be easily used in downstream tasks, like editing or analysis. Our framework offers a promising approach for discovering recurring visual patterns in image collections without supervision.

Keywords: Deep Learning · Self-Supervised Learning · Graph Learning

Author Information

Dmitriy Smirnov (MIT)
MICHAEL GHARBI (Massachusetts Institute of Technology)
Matthew Fisher (Adobe Research)
Vitor Guizilini (Toyota Research Institute)
Alexei Efros (UC Berkeley)
Justin Solomon (MIT)

More from the Same Authors