Timezone: »
In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement. Among these algorithms, deep unfolding methods demonstrate promising performance but suffer from two issues. Firstly, they do not estimate the degradation patterns and ill-posedness degree from CASSI to guide the iterative learning. Secondly, they are mainly CNN-based, showing limitations in capturing long-range dependencies. In this paper, we propose a principled Degradation-Aware Unfolding Framework (DAUF) that estimates parameters from the compressed image and physical mask, and then uses these parameters to control each iteration. Moreover, we customize a novel Half-Shuffle Transformer (HST) that simultaneously captures local contents and non-local dependencies. By plugging HST into DAUF, we establish the first Transformer-based deep unfolding method, Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST), for HSI reconstruction. Experiments show that DAUHST surpasses state-of-the-art methods while requiring cheaper computational and memory costs. Code and models are publicly available at https://github.com/caiyuanhao1998/MST
Author Information
Yuanhao Cai (Tsinghua Shenzhen International Graduate School)
Jing Lin
Haoqian Wang (Tsinghua Shenzhen International Graduate School)
Xin Yuan (Westlake University)
Henghui Ding (Swiss Federal Institute of Technology)
Yulun Zhang (ETH Zürich)
Radu Timofte (Bayerische Julius-Maximilians-Universität Würzburg)
Luc V Gool (Computer Vision Lab, ETH Zurich)
More from the Same Authors
-
2019 Poster: Gated CRF Loss for Weakly Supervised Semantic Image Segmentation »
Anton Obukhov · Stamatios Georgoulis · Dengxin Dai · Luc V Gool -
2021 Spotlight: Aligned Structured Sparsity Learning for Efficient Image Super-Resolution »
Yulun Zhang · Huan Wang · Can Qin · Yun Fu -
2021 : Spatial-Temporal Gated Transformersfor Efficient Video Processing »
Yawei Li · Babak Ehteshami Bejnordi · Bert Moons · Tijmen Blankevoort · Amirhossein Habibian · Radu Timofte · Luc V Gool -
2022 Poster: Effective Backdoor Defense by Exploiting Sensitivity of Poisoned Samples »
Weixin Chen · Baoyuan Wu · Haoqian Wang -
2022 Poster: Cross Aggregation Transformer for Image Restoration »
Zheng Chen · Yulun Zhang · Jinjin Gu · yongbing zhang · Linghe Kong · Xin Yuan -
2022 Poster: Recurrent Video Restoration Transformer with Guided Deformable Attention »
Jingyun Liang · Yuchen Fan · Xiaoyu Xiang · Rakesh Ranjan · Eddy Ilg · Simon Green · Jiezhang Cao · Kai Zhang · Radu Timofte · Luc V Gool -
2023 Poster: Binarized Spectral Compressive Imaging »
Yuanhao Cai · Yuxin Zheng · Jing Lin · Haoqian Wang · Xin Yuan · Yulun Zhang -
2023 Poster: LART: Neural Correspondence Learning with Latent Regularization Transformer for 3D Motion Transfer »
Haoyu Chen · Hao Tang · Radu Timofte · Luc V Gool · Guoying Zhao -
2023 Poster: Autodecoding Latent 3D Diffusion Models »
Evangelos Ntavelis · Aliaksandr Siarohin · Kyle Olszewski · Chaoyang Wang · Luc V Gool · Sergey Tulyakov -
2023 Poster: Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding »
Zhejun Zhang · Alexander Liniger · Christos Sakaridis · Fisher Yu · Luc V Gool -
2023 Poster: Hierarchical Integration Diffusion Model for Realistic Image Deblurring »
Zheng Chen · Yulun Zhang · Ding Liu · bin xia · Jinjin Gu · Linghe Kong · Xin Yuan -
2023 Poster: Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping »
Chunming He · Kai Li · Yachao Zhang · Guoxia Xu · Longxiang Tang · Yulun Zhang · Zhenhua Guo · Xiu Li -
2023 Poster: SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Process »
Mengyu Wang · Henghui Ding · Jun Hao Liew · Jiajun Liu · Yao Zhao · Yunchao Wei -
2023 Poster: QuantSR: Accurate Low-bit Quantization for Efficient Image Super-Resolution »
Haotong Qin · Yulun Zhang · Yifu Ding · Yifan liu · Xianglong Liu · Martin Danelljan · Fisher Yu -
2023 Poster: Revisiting Evaluation Metrics for Semantic Segmentation: Optimization and Evaluation of Fine-grained Intersection over Union »
Zifu Wang · Maxim Berman · Amal Rannen-Triki · Philip Torr · Devis Tuia · Tinne Tuytelaars · Luc V Gool · Jiaqian Yu · Matthew Blaschko -
2023 Poster: Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset »
Jing Lin · Ailing Zeng · Shunlin Lu · Yuanhao Cai · Ruimao Zhang · Haoqian Wang · Lei Zhang -
2022 Spotlight: Lightning Talks 6B-3 »
Lingfeng Yang · Yao Lai · Zizheng Pan · Zhenyu Wang · Weicong Liang · Chuanyang Zheng · Jian-Wei Zhang · Peng Jin · Jing Liu · Xiuying Wei · Yao Mu · Xiang Li · YUHUI YUAN · Zizheng Pan · Yifan Sun · Yunchen Zhang · Jianfei Cai · Hao Luo · zheyang li · Jinfa Huang · Haoyu He · Yi Yang · Ping Luo · Fenglin Liu · Henghui Ding · Borui Zhao · Xiangguo Zhang · Kai Zhang · Pichao WANG · Bohan Zhuang · Wei Chen · Ruihao Gong · Zhi Yang · Xian Wu · Feng Ding · Jianfei Cai · Xiao Luo · Renjie Song · Weihong Lin · Jian Yang · Wenming Tan · Bohan Zhuang · Shanghang Zhang · Shen Ge · Fan Wang · Qi Zhang · Guoli Song · Jun Xiao · Hao Li · Ding Jia · David Clifton · Ye Ren · Fengwei Yu · Zheng Zhang · Jie Chen · Shiliang Pu · Xianglong Liu · Chao Zhang · Han Hu -
2022 Spotlight: Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning »
Weicong Liang · YUHUI YUAN · Henghui Ding · Xiao Luo · Weihong Lin · Ding Jia · Zheng Zhang · Chao Zhang · Han Hu -
2022 Spotlight: Lightning Talks 5A-4 »
Yangrui Chen · Zhiyang Chen · Liang Zhang · Hanqing Wang · Jiaqi Han · Shuchen Wu · shaohui peng · Ganqu Cui · Yoav Kolumbus · Noemi Elteto · Xing Hu · Anwen Hu · Wei Liang · Cong Xie · Lifan Yuan · Noam Nisan · Wenbing Huang · Yousong Zhu · Ishita Dasgupta · Luc V Gool · Tingyang Xu · Rui Zhang · Qin Jin · Zhaowen Li · Meng Ma · Bingxiang He · Yangyi Chen · Juncheng Gu · Wenguan Wang · Ke Tang · Yu Rong · Eric Schulz · Fan Yang · Wei Li · Zhiyuan Liu · Jiaming Guo · Yanghua Peng · Haibin Lin · Haixin Wang · Qi Yi · Maosong Sun · Ruizhi Chen · Chuan Wu · Chaoyang Zhao · Yibo Zhu · Liwei Wu · xishan zhang · Zidong Du · Rui Zhao · Jinqiao Wang · Ling Li · Qi Guo · Ming Tang · Yunji Chen -
2022 Spotlight: Lightning Talks 5B-4 »
Yuezhi Yang · Zeyu Yang · Yong Lin · Yishi Xu · Linan Yue · Tao Yang · Weixin Chen · Qi Liu · Jiaqi Chen · Dongsheng Wang · Baoyuan Wu · Yuwang Wang · Hao Pan · Shengyu Zhu · Zhenwei Miao · Yan Lu · Lu Tan · Bo Chen · Yichao Du · Haoqian Wang · Wei Li · Yanqing An · Ruiying Lu · Peng Cui · Nanning Zheng · Li Wang · Zhibin Duan · Xiatian Zhu · Mingyuan Zhou · Enhong Chen · Li Zhang -
2022 Spotlight: Towards Versatile Embodied Navigation »
Hanqing Wang · Wei Liang · Luc V Gool · Wenguan Wang -
2022 Spotlight: Effective Backdoor Defense by Exploiting Sensitivity of Poisoned Samples »
Weixin Chen · Baoyuan Wu · Haoqian Wang -
2022 Spotlight: Recurrent Video Restoration Transformer with Guided Deformable Attention »
Jingyun Liang · Yuchen Fan · Xiaoyu Xiang · Rakesh Ranjan · Eddy Ilg · Simon Green · Jiezhang Cao · Kai Zhang · Radu Timofte · Luc V Gool -
2022 Spotlight: Cross Aggregation Transformer for Image Restoration »
Zheng Chen · Yulun Zhang · Jinjin Gu · yongbing zhang · Linghe Kong · Xin Yuan -
2022 Spotlight: Lightning Talks 2B-1 »
Yehui Tang · Jian Wang · Zheng Chen · man zhou · Peng Gao · Chenyang Si · SHANGKUN SUN · Yixing Xu · Weihao Yu · Xinghao Chen · Kai Han · Hu Yu · Yulun Zhang · Chenhui Gou · Teli Ma · Yuanqi Chen · Yunhe Wang · Hongsheng Li · Jinjin Gu · Jianyuan Guo · Qiman Wu · Pan Zhou · Yu Zhu · Jie Huang · Chang Xu · Yichen Zhou · Haocheng Feng · Guodong Guo · yongbing zhang · Ziyi Lin · Feng Zhao · Ge Li · Junyu Han · Jinwei Gu · Jifeng Dai · Chao Xu · Xinchao Wang · Linghe Kong · Shuicheng Yan · Yu Qiao · Chen Change Loy · Xin Yuan · Errui Ding · Yunhe Wang · Deyu Meng · Jingdong Wang · Chongyi Li -
2022 Poster: I2DFormer: Learning Image to Document Attention for Zero-Shot Image Classification »
Muhammad Ferjad Naeem · Yongqin Xian · Luc V Gool · Federico Tombari -
2022 Poster: Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning »
Weicong Liang · YUHUI YUAN · Henghui Ding · Xiao Luo · Weihong Lin · Ding Jia · Zheng Zhang · Chao Zhang · Han Hu -
2022 Poster: Towards Versatile Embodied Navigation »
Hanqing Wang · Wei Liang · Luc V Gool · Wenguan Wang -
2021 Poster: Slow Learning and Fast Inference: Efficient Graph Similarity Computation via Knowledge Distillation »
Can Qin · Handong Zhao · Lichen Wang · Huan Wang · Yulun Zhang · Yun Fu -
2021 Poster: Aligned Structured Sparsity Learning for Efficient Image Super-Resolution »
Yulun Zhang · Huan Wang · Can Qin · Yun Fu -
2021 Poster: Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations »
Wouter Van Gansbeke · Simon Vandenhende · Stamatios Georgoulis · Luc V Gool -
2021 Poster: Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training »
Yuanhao Cai · Xiaowan Hu · Haoqian Wang · Yulun Zhang · Hanspeter Pfister · Donglai Wei -
2020 Poster: Neural Sparse Representation for Image Restoration »
Yuchen Fan · Jiahui Yu · Yiqun Mei · Yulun Zhang · Yun Fu · Ding Liu · Thomas Huang -
2020 Poster: GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network »
Prune Truong · Martin Danelljan · Luc V Gool · Radu Timofte -
2020 Poster: Soft Contrastive Learning for Visual Localization »
Janine Thoma · Danda Pani Paudel · Luc V Gool -
2017 Poster: Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations »
Eirikur Agustsson · Fabian Mentzer · Michael Tschannen · Lukas Cavigelli · Radu Timofte · Luca Benini · Luc V Gool -
2016 Poster: Dynamic Filter Networks »
Xu Jia · Bert De Brabandere · Tinne Tuytelaars · Luc V Gool -
2014 Poster: Quantized Kernel Learning for Feature Matching »
Danfeng Qin · Xuanli Chen · Matthieu Guillaumin · Luc V Gool -
2014 Poster: Self-Adaptable Templates for Feature Coding »
Xavier Boix · Gemma Roig · Salomon Diether · Luc V Gool -
2011 Poster: Learning Probabilistic Non-Linear Latent Variable Models for Tracking Complex Activities »
Angela Yao · Juergen Gall · Luc V Gool · Raquel Urtasun