Timezone: »
Implicit neural representations are a promising new avenue of representing general signals by learning a continuous function that, parameterized as a neural network, maps the domain of a signal to its codomain; the mapping from spatial coordinates of an image to its pixel values, for example. Being capable of conveying fine details in a high dimensional signal, unboundedly of its domain, implicit neural representations ensure many advantages over conventional discrete representations. However, the current approach is difficult to scale for a large number of signals or a data set, since learning a neural representation---which is parameter heavy by itself---for each signal individually requires a lot of memory and computations. To address this issue, we propose to leverage a meta-learning approach in combination with network compression under a sparsity constraint, such that it renders a well-initialized sparse parameterization that evolves quickly to represent a set of unseen signals in the subsequent training. We empirically demonstrate that meta-learned sparse neural representations achieve a much smaller loss than dense meta-learned models with the same number of parameters, when trained to fit each signal using the same number of optimization steps.
Author Information
Jaeho Lee (KAIST)
Jihoon Tack (KAIST)
Namhoon Lee (University of Oxford)
Jinwoo Shin (KAIST)
More from the Same Authors
-
2021 : SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning »
Jongjin Park · Younggyo Seo · Jinwoo Shin · Honglak Lee · Pieter Abbeel · Kimin Lee -
2022 : STUNT: Few-shot Tabular Learning with Self-generated Tasks from Unlabeled Tables »
Jaehyun Nam · Jihoon Tack · Kyungmin Lee · Hankook Lee · Jinwoo Shin -
2022 : Dynamics-Augmented Decision Transformer for Offline Dynamics Generalization »
Changyeon Kim · Junsu Kim · Younggyo Seo · Kimin Lee · Honglak Lee · Jinwoo Shin -
2022 : Unsupervised Meta-learning via Few-shot Pseudo-supervised Contrastive Learning »
Huiwon Jang · Hankook Lee · Jinwoo Shin -
2023 Poster: Multi-scale Diffusion Denoised Smoothing »
Jongheon Jeong · Jinwoo Shin -
2023 Poster: Slimmed Asymmetrical Contrastive Learning and Cross Distillation for Lightweight Model Training »
Jian Meng · Li Yang · Kyungmin Lee · Jinwoo Shin · Deliang Fan · Jae-sun Seo -
2023 Poster: Modality-Agnostic Self-Supervised Learning with Meta-Learned Masked Auto-Encoder »
Huiwon Jang · Jihoon Tack · Daewon Choi · Jongheon Jeong · Jinwoo Shin -
2023 Poster: Learning Large-scale Neural Fields via Context Pruned Meta-Learning »
Jihoon Tack · Subin Kim · Sihyun Yu · Jaeho Lee · Jinwoo Shin · Jonathan Richard Schwarz -
2023 Poster: S-CLIP: Semi-supervised Vision-Language Pre-training using Few Specialist Captions »
Sangwoo Mo · Minkyu Kim · Kyungmin Lee · Jinwoo Shin -
2023 Poster: Guide Your Agent with Adaptive Multimodal Rewards »
Changyeon Kim · Younggyo Seo · Hao Liu · Lisa Lee · Jinwoo Shin · Honglak Lee · Kimin Lee -
2023 Poster: Accelerating Reinforcement Learning with Value-Conditional State Entropy Exploration »
Dongyoung Kim · Jinwoo Shin · Pieter Abbeel · Younggyo Seo -
2023 Poster: Collaborative Score Distillation for Consistent Visual Synthesis »
Subin Kim · Kyungmin Lee · June Suk Choi · Jongheon Jeong · Kihyuk Sohn · Jinwoo Shin -
2022 Poster: NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation »
Taesik Gong · Jongheon Jeong · Taewon Kim · Yewon Kim · Jinwoo Shin · Sung-Ju Lee -
2022 Poster: RényiCL: Contrastive Representation Learning with Skew Rényi Divergence »
Kyungmin Lee · Jinwoo Shin -
2022 Poster: Meta-Learning with Self-Improving Momentum Target »
Jihoon Tack · Jongjin Park · Hankook Lee · Jaeho Lee · Jinwoo Shin -
2022 Poster: Scalable Neural Video Representations with Learnable Positional Features »
Subin Kim · Sihyun Yu · Jaeho Lee · Jinwoo Shin -
2021 Poster: Improving Transferability of Representations via Augmentation-Aware Self-Supervision »
Hankook Lee · Kibok Lee · Kimin Lee · Honglak Lee · Jinwoo Shin -
2021 Poster: Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning »
Junsu Kim · Younggyo Seo · Jinwoo Shin -
2021 Poster: RoMA: Robust Model Adaptation for Offline Model-based Optimization »
Sihyun Yu · Sungsoo Ahn · Le Song · Jinwoo Shin -
2021 Poster: Scaling Neural Tangent Kernels via Sketching and Random Features »
Amir Zandieh · Insu Han · Haim Avron · Neta Shoham · Chaewon Kim · Jinwoo Shin -
2021 Poster: Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning »
Jongjin Park · Younggyo Seo · Chang Liu · Li Zhao · Tao Qin · Jinwoo Shin · Tie-Yan Liu -
2021 Poster: Object-aware Contrastive Learning for Debiased Scene Representation »
Sangwoo Mo · Hyunwoo Kang · Kihyuk Sohn · Chun-Liang Li · Jinwoo Shin -
2021 Poster: SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustness »
Jongheon Jeong · Sejun Park · Minkyu Kim · Heung-Chang Lee · Do-Guk Kim · Jinwoo Shin -
2020 Poster: Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning »
Jaehyung Kim · Youngbum Hur · Sejun Park · Eunho Yang · Sung Ju Hwang · Jinwoo Shin -
2020 Poster: Time-Reversal Symmetric ODE Network »
In Huh · Eunho Yang · Sung Ju Hwang · Jinwoo Shin -
2020 Poster: Learning from Failure: De-biasing Classifier from Biased Classifier »
Junhyun Nam · Hyuntak Cha · Sungsoo Ahn · Jaeho Lee · Jinwoo Shin -
2020 Poster: Adversarial Self-Supervised Contrastive Learning »
Minseon Kim · Jihoon Tack · Sung Ju Hwang -
2020 Poster: CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances »
Jihoon Tack · Sangwoo Mo · Jongheon Jeong · Jinwoo Shin -
2020 Poster: Guiding Deep Molecular Optimization with Genetic Exploration »
Sungsoo Ahn · Junsu Kim · Hankook Lee · Jinwoo Shin -
2020 Poster: Consistency Regularization for Certified Robustness of Smoothed Classifiers »
Jongheon Jeong · Jinwoo Shin -
2020 Poster: Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning »
Younggyo Seo · Kimin Lee · Ignasi Clavera Gilaberte · Thanard Kurutach · Jinwoo Shin · Pieter Abbeel -
2020 Poster: Learning Bounds for Risk-sensitive Learning »
Jaeho Lee · Sejun Park · Jinwoo Shin -
2020 Poster: Few-shot Visual Reasoning with Meta-Analogical Contrastive Learning »
Youngsung Kim · Jinwoo Shin · Eunho Yang · Sung Ju Hwang -
2019 Poster: Mining GOLD Samples for Conditional GANs »
Sangwoo Mo · Chiheon Kim · Sungwoong Kim · Minsu Cho · Jinwoo Shin -
2018 Poster: A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks »
Kimin Lee · Kibok Lee · Honglak Lee · Jinwoo Shin -
2018 Poster: Stochastic Chebyshev Gradient Descent for Spectral Optimization »
Insu Han · Haim Avron · Jinwoo Shin -
2018 Spotlight: A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks »
Kimin Lee · Kibok Lee · Honglak Lee · Jinwoo Shin -
2018 Spotlight: Stochastic Chebyshev Gradient Descent for Spectral Optimization »
Insu Han · Haim Avron · Jinwoo Shin -
2018 Poster: Learning to Specialize with Knowledge Distillation for Visual Question Answering »
Jonghwan Mun · Kimin Lee · Jinwoo Shin · Bohyung Han -
2017 Poster: Gauging Variational Inference »
Sungsoo Ahn · Michael Chertkov · Jinwoo Shin -
2016 Poster: Synthesis of MCMC and Belief Propagation »
Sungsoo Ahn · Michael Chertkov · Jinwoo Shin -
2016 Oral: Synthesis of MCMC and Belief Propagation »
Sungsoo Ahn · Michael Chertkov · Jinwoo Shin -
2015 Poster: Minimum Weight Perfect Matching via Blossom Belief Propagation »
Sungsoo Ahn · Sejun Park · Michael Chertkov · Jinwoo Shin -
2015 Spotlight: Minimum Weight Perfect Matching via Blossom Belief Propagation »
Sungsoo Ahn · Sejun Park · Michael Chertkov · Jinwoo Shin -
2013 Poster: A Graphical Transformation for Belief Propagation: Maximum Weight Matchings and Odd-Sized Cycles »
Jinwoo Shin · Andrew E Gelfand · Misha Chertkov