Skip to yearly menu bar Skip to main content


NeurIPS 2024 Call for Ethics Reviewers

 

We are seeking qualified AI ethics reviewers who can assess NeurIPS submissions in at least one of the following areas:

  • Research involving human subjects
  • Data privacy, copyright, and consent
  • Data quality and representativeness
  • Safety and security
  • Discrimination, bias, and fairness
  • Deception and harassment
  • Environmental Impact
  • Human rights (including surveillance)

NeurIPS has asked authors to consider ethics and broader impact when submitting their papers since 2021, and adopted a Code of Ethics in April 2023. Ethics reviews are a second round of review that take place should the program committee flag any potential concerns during the technical review phase. Ethics reviewers provide feedback to the program committee regarding risks and harms of the work, and recommend potential mitigations prior to acceptance. To minimize exposure risks, ethics reviews are only published at the author’s discretion; unlike the main reviews, which are all public. Please refer below and to the Ethics Review Guidelines for further details.

We ask that ethics reviewers:

  • Review up to 3 papers each,
  • Provide ethics reviews during at least one of:
    • The period of July 15 - 26, 2024 for submissions to the main conference track, and/or
    • The period of July 26 - August 6, 2024, for submissions to the Datasets & Benchmarks track, and/or
    • The period of August 7-30, 2024, for emergency reviews to cover late review requests, and
  • Participate asynchronously in any relevant follow-up discussions with the authors and program committee during the author response period of August 2024. 
 

If you are able and willing to participate in the review process, please sign up at this Google form (http://tiny.cc/neurips2024ethics). Feel free to share this call with your colleagues. 

Thank you for your consideration,

2024 ETHICS REVIEW COMMITTEE CO-CHAIRS

Jiahao Chen, Director of AI/ML, New York City Office of Technology and Innovation
Hima Lakkaraju, Assistant Professor, Harvard University
ethics-review-chairs@neurips.cc

 

Key dates for ethics review

A full list of relevant dates for the conference is available here.

 

[Main] refers to the main conference track; [D&B] refers to Datasets and Benchmarks 

 
  • [Main] Paper submission deadline May 22 '24 08:00 PM UTC
  • [D&B] Paper submission deadline Jun 05 '24 (Anywhere on Earth)
  • [Main] End of main review period Jul 12 ‘24 (Anywhere on Earth)
  • [Main] Beginning of ethics review process Jul 16 ‘24 (Anywhere on Earth)
  • [D&B] End of main review period Jul 25 ‘24 (Anywhere on Earth)
  • [Main] End of ethics review process Jul 26 ‘24 (Anywhere on Earth)
  • [D&B] Beginning of ethics review process Jul 26 ‘24 (Anywhere on Earth)
  • [D&B] End of ethics review process Aug 6 ‘24 (Anywhere on Earth)
  • [D&B] Start of Author discussions Aug 07 '24 (Anywhere on Earth)
  • [D&B] End of author/reviewer discussions: Aug 30 '24 (Anywhere on Earth)
  • Author notifications: Sep 26 '24 01:00 AM UTC
 

About the ethics review process

The main reviews conducted through the program committee (reviewers, program chairs and area chairs) is, and continues to be, the sole decision making process for accepting or rejecting papers for publications at NeurIPS. Reviewers are expected to review submissions not just for pure technical merit, but also in the context of the NeuriPS Code of Ethics.

The ethics review is a second round of review that takes place only when the program committee flags any potential concerns during the technical review phase that merits further attention. Ethics reviewers provide feedback to the program committee regarding risks and harms of the work in line with the NeurIPS Code of Ethics, and recommend potential mitigations that can be provided to authors to incorporate as feedback to revise their submissions.

The ethics review process is not a disciplinary or punitive process. However, in rare situations, the NeurIPS program committee may decide to reject submissions that have grossly violated the NeurIPS Code of Ethics, taking into account recommendations from the ethics reviews. Even so, the authors were provided with substantial guidance and relevant citations, and were invited to revise and resubmit to NeurIPS.

As detailed in the Ethics Guidelines for Reviewers, the ethics reviews generally follow the double-blinded review process of the main technical reviews. ​However, additional steps are taken in order to minimize exposure risks. During the ethics review process, any submissions flagged for ethics review will not be publicly labeled as such. During the author response period, ethics reviews will be anonymized when made visible to authors and technical reviewers. When the final accept or reject decision has been made, authors who have accepted papers may, at their discretion, choose to make their ethics review public.

Disclaimer. The views expressed here do not necessarily represent the views of the City of New York or its Office of Technology and Innovation.