Timezone: »

 
Poster
SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery
Yezhen Cong · Samar Khanna · Chenlin Meng · Patrick Liu · Erik Rozi · Yutong He · Marshall Burke · David Lobell · Stefano Ermon

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #439
in Poster Session 4 »
Unsupervised pre-training methods for large vision models have shown to enhance performance on downstream supervised tasks. Developing similar techniques for satellite imagery presents significant opportunities as unlabelled data is plentiful and the inherent temporal and multi-spectral structure provides avenues to further improve existing pre-training strategies. In this paper, we present SatMAE, a pre-training framework for temporal or multi-spectral satellite imagery based on Masked Autoencoder (MAE). To leverage temporal information, we include a temporal embedding along with independently masking image patches across time. In addition, we demonstrate that encoding multi-spectral data as groups of bands with distinct spectral positional encodings is beneficial. Our approach yields strong improvements over previous state-of-the-art techniques, both in terms of supervised learning performance on benchmark datasets (up to $\uparrow$ 7%), and transfer learning performance on downstream remote sensing tasks, including land cover classification (up to $\uparrow$ 14%) and semantic segmentation. Code and data are available on the project website: https://sustainlab-group.github.io/SatMAE/
Keywords: Self-supervised learning · Masked Autoencoder · Satellite imagery
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Yezhen Cong (Stanford University)
Samar Khanna (Stanford University)
Chenlin Meng (Stanford University)
Patrick Liu (Stanford University)
Erik Rozi (Stanford University)
Yutong He (Carnegie Mellon University)
Marshall Burke (Stanford University)
David Lobell (Stanford University)
Stefano Ermon (Stanford)

More from the Same Authors