Presentation
The CodaLab competition platform and its new version Codabench, released in August 2023, are powerful open-source portals with broad international visibility. This platform hosts scientific competitions and benchmarks, allowing the submission of results (like any competition platform) as well as code (unlike Google's Kaggle platform). You can either take part in an existing competition (or benchmark), or organize your own.
Most of the competitions hosted on such platforms are data science machine learning competitions, organized by companies or research organizations, but they are not limited to this field of application. Codalab can also accommodate any problem for which a quantitative solution can be entrusted to an evaluation program provided by the organizers. The evaluation program must return a numerical score, which is displayed on a leaderboard where participants' performances are compared.
Codalab has been ranked first in a recent survey of machine learning competition platforms.
Technical description of the Codalab production infrastructure
Main historical milestones of Codalab
- 2013: Microsoft open-sources Codalab Medical data. Result submission.
- 2014: Computer vision, speech, NLP, IR. MSCOCO: 361 participants.
- 2015: AutoML: 687 participants Hackathons. Coopetitions.
- 2016: Codalab competitions to U. PSaclay. 480 challenges, 10000 users.
- 2018: Google sponsors AutoDL series.
- 2019: RTE organizes L2RPN challenge.
- 2020: 50,000 users, 1000 competitions in total.
- 2021: on-premise storage solution to replace private storage provider. 50 competitions/month. 20 GPUs
- 2023: Codabench for benchmarks, successor of Codalab, more than 100 competitions and 2500 new users monthly.
The current hardware of the project (150 k€ invested since 2018) has been mainly funded by research contracts (Digiteo, Région Ile-de-France, ANR) and by University Paris-Saclay, Inria and CNRS. The sponsor list having contributed to Codalab since its creation contains also Microsoft, Google, 4Paradigm, universities of Stanford and Barcelona, European EIT Health.
Platform users and Significant realisations
Statistics of IP connections are spread all over the world, one third on each continent Europe, America and Asia.
