Instruction Guide: Datasets & Benchmarks Track Dataset Hosting Platforms 

 

This webpage serves as an instruction guide for authors making a submission to the Datasets & Benchmarks Track CFP. At a high level, authors of datasets will be required to make their dataset available along with Croissant machine-readable metadata in order to streamline the review process and meet industry standards for high quality documentation, reproducibility, and accessibility. As a part of these new requirements, we are recommending use of preferred hosting platforms–Dataverse, Kaggle, Hugging Face, and OpenML–which will make compliance simple for authors and reviewers. For more detailed information regarding the rationale for submission requirements new in 2025, please read our blog post. 

The dataset hosting process as part of submitting to the Datasets & Benchmarks Track involves:

1. Choosing among 4 options to host your dataset: Harvard Dataverse, Kaggle, Hugging Face, and OpenML
2. Using platform tooling to download the automatically generated Croissant file
3. Including a URL to your dataset and uploading the generated Croissant file in Open Review
4. If your submission is accepted: making your dataset public by the camera ready deadline

 

Choose a Preferred Hosting Platforms

Harvard Dataverse, Kaggle, Hugging Face, and OpenML platforms are the preferred hosting platforms for datasets. These platforms automatically generate a Croissant file and allow for programmatic metadata verification and dataset assessment which will streamline and standardize the review process. When you make your dataset accessible via one of these platforms, making a submission will be as simple as providing a URL to the dataset and uploading a generated Croissant file.

The table below outlines key platform features to help authors choose where to host their dataset. Authors may make their dataset accessible via more than one platform at any time.

 

Authors are responsible for reviewing and complying with the Terms of Service of the platform(s) they choose to use.

 

Alternative: Self-hosting your Dataset and Other Data Storage Platforms

If you choose NOT to release your dataset via one of these preferred platforms, you can self-host the data or use other platforms, but you will still be required to make your dataset accessible to reviewers via URL (e.g., to a GitHub repo, public cloud bucket, Zenodo, etc.) and manually generate and upload a Croissant file representation of your dataset as part of the Open Review submission process. This Python tutorial demonstrates how to generate a Croissant file. Find more documentation on the Croissant GitHub repository.

 

How to Publish on Preferred Hosting Platforms

This section provides specific guidance and documentation on how to make your dataset and its Croissant metadata file accessible via the preferred hosting platforms: Harvard Dataverse, Kaggle, Hugging Face, and OpenML.

 

 

 

What to Include in Your Submission

When you submit to Open Review, you will be required to provide:

1. A URL to your dataset accessible to reviewers
2. A Croissant metadata file
3. A confirmation that you verified the validity of your Croissant file.

Please use this online tool to verify your Croissant file. This tool will check whether it is valid, sufficiently complete, and whether the data can be automatically accessed and loaded (the latter may not be possible for all datasets). The same information will be made available to reviewers. 

After submissions are closed, if your Croissant file is invalid or if your data is not accessible, your submission may be desk-rejected. Otherwise, reviewers will commence review of your submission paper, dataset, and metadata. 

If your dataset is accepted, you will be required to make it public by the camera ready deadline. Failure to do so may result in removal from the conference and proceedings.

 

FAQ

Please see the Datasets and Benchmark track FAQs.