Timezone: »

ImageNet: Past, Present, and Future
Zeynep Akata · Lucas Beyer · Sanghyuk Chun · A. Sophia Koepke · Diane Larlus · Seong Joon Oh · Rafael Rezende · Sangdoo Yun · Xiaohua Zhai

Mon Dec 13 04:00 AM -- 05:15 PM (PST) @ None
Event URL: https://sites.google.com/view/imagenet-workshop/ »

Since its release in 2010, ImageNet has played an instrumental role in the development of deep learning architectures for computer vision, enabling neural networks to greatly outperform hand-crafted visual representations. ImageNet also quickly became the go-to benchmark for model architectures and training techniques which eventually reach far beyond image classification. Today’s models are getting close to “solving” the benchmark. Models trained on ImageNet have been used as strong initialization for numerous downstream tasks. The ImageNet dataset has even been used for tasks going way beyond its initial purpose of training classification model. It has been leveraged and reinvented for tasks such as few-shot learning, self-supervised learning and semi-supervised learning. Interesting re-creation of the ImageNet benchmark enables the evaluation of novel challenges like robustness, bias, or concept generalization. More accurate labels have been provided. About 10 years later, ImageNet symbolizes a decade of staggering advances in computer vision, deep learning, and artificial intelligence.

We believe now is a good time to discuss what’s next: Did we solve ImageNet? What are the main lessons learnt thanks to this benchmark? What should the next generation of ImageNet-like benchmarks encompass? Is language supervision a promising alternative? How can we reflect on the diverse requirements for good datasets and models, such as fairness, privacy, security, generalization, scale, and efficiency?

Mon 4:00 a.m. - 4:30 a.m.
(Opening presentation)

Opening ceremony

Mon 4:30 a.m. - 5:00 a.m.

Talk by Olga Russakovsky, Princeton University.

Olga Russakovsky · Kaiyu Yang
Mon 5:00 a.m. - 5:30 a.m.

Talk by Vittorio Ferrari, Google.

Vittorio Ferrari
Mon 5:30 a.m. - 6:00 a.m.

Talk by Matthias Bethge, University of Tübingen.

Matthias Bethge
Mon 6:00 a.m. - 7:00 a.m.
Live panel: The future of ImageNet (Live panel)
Matthias Bethge · Vittorio Ferrari · Olga Russakovsky
Mon 7:30 a.m. - 7:45 a.m.
Oral 1 (Oral session)
Mon 7:45 a.m. - 8:45 a.m.
Poster session 1 (Poster session) [ Visit Poster at Spot A1 in Virtual World ]  link »
Mon 8:45 a.m. - 9:15 a.m.

Talk by Rebecca Roelofs, Google.

Rebecca Roelofs
Mon 9:15 a.m. - 9:45 a.m.

Talk by Shibani Santurkar, MIT.

Shibani Santurkar
Mon 9:45 a.m. - 10:15 a.m.

We ask whether recent progress on the ImageNet classification benchmark continues to represent meaningful generalization, or whether the community has started to overfit to the idiosyncrasies of its labeling procedure. We therefore develop a significantly more robust procedure for collecting human annotations of the ImageNet validation set. Using these new labels, we reassess the accuracy of recently proposed ImageNet classifiers, and find their gains to be substantially smaller than those reported on the original labels. Furthermore, we find the original ImageNet labels to no longer be the best predictors of this independently-collected set, indicating that their usefulness in evaluating vision models may be nearing an end. Nevertheless, we find our annotation procedure to have largely remedied the errors in the original labels, reinforcing ImageNet as a powerful benchmark for future research in visual recognition.

Alexander Kolesnikov
Mon 10:15 a.m. - 11:15 a.m.
Live panel: Did we solve ImageNet? (Live panel)
Shibani Santurkar · Alexander Kolesnikov · Rebecca Roelofs
Mon 11:45 a.m. - 12:15 p.m.

[Future] Talk by Yixuan Li, U. of Wisconsin

Sharon Li
Mon 12:15 p.m. - 12:45 p.m.

[Present] Talk by Ross Wightman, Angel investor

Ross Wightman
Mon 12:45 p.m. - 1:15 p.m.

Talk by Dawn Song and Dan Hendricks, University of California, Berkeley.

Dawn Song · Dan Hendrycks
Mon 1:15 p.m. - 2:15 p.m.
Live panel: Perspectives on ImageNet. (Live panel)
Dawn Song · Ross Wightman · Sharon Li
Mon 2:15 p.m. - 2:45 p.m.
TBD (Talk)
Emily Denton · Alex Hanna
Mon 2:45 p.m. - 3:15 p.m.
Live Panel: TBD (Live panel)
Emily Denton · Alex Hanna
Mon 3:45 p.m. - 4:00 p.m.
Oral 2 (Oral session)
Mon 4:00 p.m. - 5:00 p.m.
Poster session 2 (Poster session) [ Visit Poster at Spot A0 in Virtual World ]  link »
Mon 5:00 p.m. - 5:15 p.m.
Closing & awards (Workshop closing)

Author Information

Zeynep Akata (University of Tübingen)
Lucas Beyer (Google Brain Zürich)
Sanghyuk Chun (NAVER AI Lab)

I'm a research scientist and tech leader at NAVER AI Lab, working on machine learning and its applications. In particular, my research interests focus on bridging the gap between two gigantic topics: reliable machine learning tasks (e.g., robustness [C3, C9, C10, W1, W3], de-biasing or domain generalization [C6, A6], uncertainty estimation [C11, A3], explainability [C5, C11, A2, A4, W2], and fair evaluation [C5, C11]) and learning with limited annotations (e.g., multi-modal learning [C11], weakly-supervised learning [C2, C3, C4, C5, C7, C8, C12, W2, W4, W5, W6, A2, A4], and self-supervised learning). I have contributed large-scale machine learning algorithms [C3, C9, C10, C13] in NAVER AI Lab as well. Prior to working at NAVER, I worked as a research engineer at the advanced recommendation team (ART) in Kakao from 2016 to 2018. I received a master's degree in Electrical Engineering from Korea Advanced Institute of Science and Technology (KAIST) in 2016. During my master's degree, I researched a scalable algorithm for robust subspace clustering (the algorithm is based on robust PCA and k-means clustering). Before my master's study, I worked at IUM-SOCIUS in 2012 as a software engineering internship. I also did a research internship at Networked and Distributed Computing System Lab in KAIST and NAVER Labs during summer 2013 and fall 2015, respectively.

A. Sophia Koepke (University of Tübingen)
Diane Larlus (NAVER LABS Europe)
Seong Joon Oh (NAVER AI Lab)
Rafael Rezende (NAVER LABS EUROPE)
Sangdoo Yun (Clova AI Research, NAVER Corp.)
Xiaohua Zhai (Google Brain)

More from the Same Authors