cTuning & MLCommons Collective Knowledge Challenges

[ Back ]

---

Reproduce and optimize TinyMLPerf inference v1.1 benchmarks

Open date: 2023 May 1

Closing date: 2023 May 19

---

Challenge

Reproduce and automate TinyMLPerf benchmarks.

Organizers

• MLCommons taskforce on automation and reproducibility
• cTuning foundation
• cKnowledge Ltd

Status

We have successfully reproduced TinyMLPerf v1.0 submission with microTVM on the STMicroelectronics NUCLEO-L4R5ZI board, automated it with the latest version of the MLCommons CM automation language, submit reproduce results to the TinyMLperf v1.1 round, and added all past TinyMLPerf results to the MLCommons CK playground for further collaborative analysis and improvement.

Pleases check our tutorial and reproducibility report: * Automate TinyMLPerf benchmark - useful for all SW/HW stacks and submission rounds. * Reproduce TinyMLPerf v1.0 submission.

TinyMLPerf v1.1 results will be published at te MLCommons CK playground in mid June 2023.

Related discussions for the future

• https://github.com/mlcommons/ck/pull/693
• https://github.com/mlcommons/ck/pull/700
• https://github.com/mlcommons/ck/pull/701
• https://github.com/mlcommons/ck/issues/606

Results

All results will be available in this GitHub repo and can be visualized and compared using the MLCommons Collective Knowledge Playground.

---

Self link