Timezone: »
Self-supervised learning is a strong way to learn useful representations from the bulk of natural data. It's suggested to be responsible for building the visual representation in humans, but the specific objective and algorithm are unknown. Currently, most self-supervised methods encourage the system to learn an invariant representation of different transformations of the same image in contrast to those of other images. However, such transformations are generally non-biologically plausible, and often consist of contrived perceptual schemes such as random cropping and color jittering. In this paper, we attempt to reconfigure these augmentations to be more biologically or perceptually plausible while still conferring the same benefits for encouraging a good representation. Critically, we find that random cropping can be substituted by cortical magnification, and saccade-like sampling of the image could also assist the representation learning. The feasibility of these transformations suggests a potential way that biological visual systems could implement self-supervision. Further, they break the widely accepted spatially-uniform processing assumption used in many computer vision algorithms, suggesting a role for spatially-adaptive computation in humans and machines alike.
Author Information
Binxu Wang (Harvard University)
David Mayo (Google)
Arturo Deza (MIT)
Andrei Barbu (MIT)
Colin Conwell (Harvard University)
More from the Same Authors
-
2021 : Towards Incorporating Rich Social Interactions Into MDPs »
Ravi Tejwani · Yen-Ling Kuo · Tianmin Shu · Bennett Stankovits · Dan Gutfreund · Josh Tenenbaum · Boris Katz · Andrei Barbu -
2022 : Neural Network Online Training with Sensitivity to Multiscale Temporal Structure »
Matt Jones · Tyler Scott · Gamaleldin Elsayed · Mengye Ren · Katherine Hermann · David Mayo · Michael Mozer -
2022 : On the Level Sets and Invariance of Neural Tuning Landscapes »
Binxu Wang · Carlos Ponce -
2022 : Towards Disentangling the Roles of Vision & Language in Aesthetic Experience with Multimodal DNNs »
Colin Conwell · Christopher Hamblin -
2022 : Workshop version: How hard are computer vision datasets? Calibrating dataset difficulty to viewing time »
David Mayo · Jesse Cummings · Xinyu Lin · Dan Gutfreund · Boris Katz · Andrei Barbu -
2022 : Image recognition time for humans predicts adversarial vulnerability for models »
David Mayo · Jesse Cummings · Xinyu Lin · Boris Katz · Andrei Barbu -
2022 : The Perceptual Primacy of Feeling: Affectless machine vision models explain the majority of variance in visually evoked affect and aesthetics »
Colin Conwell -
2021 : What can 5.17 billion regression fits tell us about artificial models of the human visual system? »
Colin Conwell · Jacob Prince · George Alvarez · Talia Konkle -
2021 : Unsupervised Representation Learning Facilitates Human-like Spatial Reasoning »
Kaushik Lakshminarasimhan · Colin Conwell -
2021 : Finding Biological Plausibility for Adversarially Robust Features via Metameric Tasks »
Anne Harrington · Arturo Deza -
2021 : Evaluating the Adversarial Robustness of a Foveated Texture Transform Module in a CNN »
Jonathan Gant · Andrzej Banburski · Arturo Deza -
2021 : What Matters In Branch Specialization? Using a Toy Task to Make Predictions »
Chenguang Li · Arturo Deza -
2021 Poster: Neural Regression, Representational Similarity, Model Zoology & Neural Taskonomy at Scale in Rodent Visual Cortex »
Colin Conwell · David Mayo · Andrei Barbu · Michael Buice · George Alvarez · Boris Katz -
2019 : Concluding Remarks & Prizes Ceremony »
Arturo Deza · Joshua Peterson · Apurva Ratan Murty · Tom Griffiths -
2019 : Poster Session »
Ethan Harris · Tom White · Oh Hyeon Choung · Takashi Shinozaki · Dipan Pal · Katherine L. Hermann · Judy Borowski · Camilo Fosco · Chaz Firestone · Vijay Veerabadran · Benjamin Lahner · Chaitanya Ryali · Fenil Doshi · Pulkit Singh · Sharon Zhou · Michel Besserve · Michael Chang · Anelise Newman · Mahesan Niranjan · Jonathon Hare · Daniela Mihai · Marios Savvides · Simon Kornblith · Christina M Funke · Aude Oliva · Virginia de Sa · Dmitry Krotov · Colin Conwell · George Alvarez · Alex Kolchinski · Shengjia Zhao · Mitchell Gordon · Michael Bernstein · Stefano Ermon · Arash Mehrjou · Bernhard Schölkopf · John Co-Reyes · Michael Janner · Jiajun Wu · Josh Tenenbaum · Sergey Levine · Yalda Mohsenzadeh · Zhenglong Zhou -
2019 : Making the next generation of machine learning datasets: ObjectNet a new object recognition benchmark »
Andrei Barbu -
2019 : Opening Remarks »
Arturo Deza · Joshua Peterson · Apurva Ratan Murty · Tom Griffiths -
2019 Workshop: Shared Visual Representations in Human and Machine Intelligence »
Arturo Deza · Joshua Peterson · Apurva Ratan Murty · Tom Griffiths -
2019 Poster: ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models »
Andrei Barbu · David Mayo · Julian Alverio · William Luo · Christopher Wang · Dan Gutfreund · Josh Tenenbaum · Boris Katz