This is the public, feature-limited version of the conference webpage. After Registration and login please visit the full version.

Workshop: ML Retrospectives, Surveys & Meta-Analyses (ML-RSA)

Chhavi Yadav, Prabhu Pradhan, Abhishek Gupta, Jesse Dodge, Mayoore Jaiswal, Peter Henderson, Ryan Lowe, Jessica Forde Jessica Forde

Fri, Dec 11th @ 16:30 GMT – Sat, Dec 12th @ 05:00 GMT
Abstract: The exponential growth of AI research has led to several papers floating on arxiv, making it difficult to review existing literature. Despite the huge demand, the proportion of survey & analyses papers published is very low due to reasons like lack of a venue and incentives. Our Workshop, ML-RSA provides a platform and incentivizes writing such types of papers. It meets the need of taking a step back, looking at the sub-field as a whole and evaluating actual progress. We will accept 3 types of papers: broad survey papers, meta-analyses, and retrospectives. Survey papers will mention and cluster different types of approaches, provide pros and cons, highlight good source code implementations, applications and emphasize impactful literature. We expect this type of paper to provide a detailed investigation of the techniques and link together themes across multiple works. The main aim of these will be to organize techniques and lower the barrier to entry for newcomers. Meta-Analyses, on the other hand, are forward-looking, aimed at providing critical insights on the current state-of-affairs of a sub-field and propose new directions based on them. These are expected to be more than just an ablation study -- though an empirical analysis is encouraged as it can provide for a stronger narrative. Ideally, they will seek to showcase trends that are not possible to be seen when looking at individual papers. Finally, retrospectives seek to provide further insights ex post by the authors of a paper: these could be technical, insights into the research process, or other helpful information that isn’t apparent from the original work.

Chat

To ask questions please use rocketchat, available only upon registration and login.

Schedule

16:30 – 16:55 GMT
Introduction
17:00 – 17:30 GMT
Invited: Shakir Mohamed
Shakir Mohamed
17:35 – 17:45 GMT
Q&A 1
Shakir Mohamed
18:00 – 18:55 GMT
Brainstorming
18:55 – 19:00 GMT
Intro to speaker 2 : Kilian Weinberger
19:00 – 19:30 GMT
Invited: Kilian Weinberger
Kilian Weinberger
19:35 – 19:45 GMT
Q&A 2
Kilian Weinberger
20:00 – 20:45 GMT
Panel
20:55 – 21:00 GMT
Intro to speaker 3 Maria De-Artega
21:00 – 21:30 GMT
Invited: Maria De-Artega
Maria De-Arteaga
21:35 – 21:45 GMT
Q&A 3
Maria De-Arteaga
21:55 – 22:00 GMT
Intro to speaker 4 : Shibani Santurkar
22:00 – 22:30 GMT
Invited: Shibani Santurkar
Shibani Santurkar
22:35 – 22:45 GMT
Q&A 4
Shibani Santurkar
22:55 – 23:00 GMT
Poster Session Starts
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Towards falsifiable interpretability research
Matthew L Leavitt
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
WILDS: A Survey and Benchmark of in-the-Wild Distribution Shifts
Pang Wei Koh
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
The Hardware Lottery
Sara Hooker
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Amandalynne Paullada
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Challenges in Deploying Machine Learning: a Survey of Case Studies
Andrei Paleyes
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
A Survey of Machine Learning Methods and Challenges for Windows Malware Classification
Edward Raff
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
Amir Karimi
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Opportunities for a More Interdisciplinary Approach to Perceptions of Fairness in Machine Learning
Sarah Brown
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Understanding Attention: In Minds and Machines
Shri Sawant
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Counterfactual Explanations for Machine Learning: A Review
Sahil Verma
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
A Brief Survey of Loop Closure Detection: A Case for Rethinking Evaluation of Intelligent Systems
Samer Nashed
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Deconstructing the Structure of Sparse Neural Networks
Maxwell D Van Gelder
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Revisiting "Qualitatively Characterizing Neural Network Optimization Problems"
Jonathan Frankle
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Feature Removal Is a Unifying Principle For Model Explanation Methods
Ian Covert
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Indic-Transformers: An Analysis of Transformer Language Models for Indian Languages
Kushal Jain
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
On Principles, Models and Methods for Learning from Irregularly Sampled Time Series...
Satya Narayan Shukla
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
A Survey of Deep Learning Approaches for OCR and Document Understanding
Nishant Subramani
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Challenging common interpretability assumptions in feature attribution explanations
Jonathan Dinu
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Beyond Methods Reproducibility in Machine Learning
Leif Hancox-Li
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Pruning Neural Networks at Initialization: Why Are We Missing the Mark?
Jonathan Frankle
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
Survey on Modeling Intensity Function of Hawkes Process Using Neural Models
Jayesh Malaviya
Fri, Dec 11th @ 23:00 GMT – Sat, Dec 12th @ 00:45 GMT
AI and the Everything in the Whole Wide World Benchmark
Deborah Raji
00:55 – 01:00 GMT
Intro to speaker 5 : Lana Sinapayen
01:00 – 01:30 GMT
Invited: Lana Sinapayen
Lana Sinapayen
01:35 – 01:45 GMT
Q&A 5
Lana Sinapayen
01:55 – 02:00 GMT
Intro to Speaker 6 : Reza Shokri
02:00 – 02:30 GMT
Invited: Reza Shokri
Reza Shokri
02:35 – 02:45 GMT
Q&A 6
Reza Shokri
03:00 – 04:00 GMT
Awardees' Talks
04:00 – 04:05 GMT
Closing