AutoML Decathlon 2022

Diverse Tasks, Modern Methods, and Efficiency at Scale

NeurIPS'22 Competition Track

Welcome to AutoML Decathlon!

The AutoML Decathlon is a competition that will evaluate the performance of participants' AutoML methods on carefully curated sets of tasks with an appropriate level of difficulty and coverage in application domain, size, and input/output characteristics.

At the start of the competition we will release 10 public development tasks that vary in their domain (including image, finance timeseries, audio, and natural sciences), problem type (including regression, single-label, and multi-label classification), and scale (ranging from several thousands to hundreds of thousands of observations). These public tasks will be representative of (but distinct from) the final set of test tasks on which participants' AutoML methods will be ultimately evaluated on at the conclusion of the competition. For more background, see our blog post.

A prize of $15,000 will be awarded to the winning team. See the competition description for details.

Final Test Phase Results

Congratulations to Team TrueFit for being the winner of AutoML Decathlon 2022!!

There were many amazing teams competing and consistently tuning their submissions; it was a great experience to hear some of the teams present on the details of their final method and overall competition experience during our recent NeurIPS Virtual Competition Workshop. A huge thank you to everyone who participated and helped make this possible! The video recording of the workshop is directly below the leaderboard.

A description of the AUP (Area Under Performance-profile curve) metric used to rank the methods.

Virtual Session Recording

Announcements

  • The competition submission deadline has been moved to Nov 10, 2022.
  • As of Sept 01, 2022, the set of development tasks has been updated. Please redownload it for the latest version.
  • The official start of the competition has been moved from Jul 13, 2022 to Jul 15, 2022.
  • We have created a new CodaLab competition page, in order to facilitate certain design/leaderboard elements. The other links on this website have been updated to reflect this as well. Please register for this new competition. We will also be reaching out to participants of the old competition to notify them. Thank you for your understanding!

Important Dates

  • Jul 15, 2022: The official start of AutoML Decathlon; the starter kit will be made available and the submission portal will be opened to participants
  • Nov 10, 2022 (11:59 pm Anywhere on Earth): The submission portal will close and we will begin evaluating final submissions
  • Dec 07, 2022 (tentative): The final results will be announced

Getting Started

Please read through the competition details on this website. Once you are ready to begin, you can download the starter kit here. Feel free to email the organizers with any questions you may have. Good luck!