Transparency

Benchmark run log

Every time we run an AI model against a benchmark set, the row is recorded here exactly as it happened. Not a rollup like the /benchmark leaderboards, but one row per call, so any summary figure on this site can be traced back to the actual run that produced it.Read more
179 test runs21 models tested6 benchmark setsMost recent run
GitHub

7 runs
  1. 21:00MiMo-V2.6-FlashArti kata Jawa sehari-hariword meaning100/135
  2. 20:36MiMo-V2.6-FlashNormalisasi teks Indonesiaopen answerBuilt-in prompt37/49
  3. 20:26MiMo-V2.6-FlashKoreksi ejaan Indonesiaopen answer26/38
  4. 20:15MiMo-V2.6-FlashTingkat tutur bahasa Jawaopen answer5/6
  5. 20:14MiMo-V2.6-FlashTingkat tutur bahasa Jawamultiple choice22/27
  6. 19:29MiMo-V2.6-FlashArti kata Jawa langkaword meaning15/43
  7. 19:27MiMo-V2.6-FlashPemahaman slang Indonesiamultiple choice40/42

8 runs
  1. 22:08DeepSeek V4.1 FlashArti kata Jawa langkaword meaning22/43
  2. 21:54DeepSeek V4.1 FlashPemahaman slang Indonesiamultiple choice41/42
  3. 21:36DeepSeek V4.1 FlashTingkat tutur bahasa Jawaopen answer6/6
  4. 21:36DeepSeek V4.1 FlashTingkat tutur bahasa Jawamultiple choice26/27
  5. 21:22Mercury 2.5Arti kata Jawa langkaword meaning13/43
  6. 21:21Mercury 2.5Tingkat tutur bahasa Jawaopen answer4/6
  7. 21:21Mercury 2.5Tingkat tutur bahasa Jawamultiple choice22/27
  8. 21:19Mercury 2.5Pemahaman slang Indonesiamultiple choice39/42

4 runs
  1. 19:51GPT-6 AstraPemahaman slang Indonesiamultiple choice42/42
  2. 10:06GPT-6 AstraTingkat tutur bahasa Jawaopen answer6/6
  3. 10:05GPT-6 AstraTingkat tutur bahasa Jawamultiple choice26/27
  4. 10:01GPT-6 AstraArti kata Jawa langkaword meaning33/43

9 runs
  1. 22:35GLM 5.3Arti kata Jawa langkaword meaning14/43
  2. 22:15GLM 5.3Arti kata Jawa sehari-hariword meaning98/135
  3. 21:45GLM 5.3Normalisasi teks Indonesiaopen answerPrompt stating the key's rule37/49
  4. 21:38GLM 5.3Normalisasi teks Indonesiaopen answerDetailed prompt forbidding word swaps37/49
  5. 21:31GLM 5.3Normalisasi teks Indonesiaopen answerBuilt-in prompt30/49
  6. 18:44GLM 5.3Koreksi ejaan Indonesiaopen answer26/38
  7. 08:30GLM 5.3Tingkat tutur bahasa Jawaopen answer3/6
  8. 08:14GLM 5.3Tingkat tutur bahasa Jawamultiple choice21/27
  9. 08:00GLM 5.3Pemahaman slang Indonesiamultiple choice40/42

2 runs
  1. 23:30GLM 5.3 FlashArti kata Jawa langkaword meaning11/43
  2. 22:44GLM 5.3 FlashArti kata Jawa sehari-hariword meaning97/135

The number at the far right of each row is the score: correct answers out of items scored. It can be smaller than the set's full item count when some items are held back or still awaiting human judgment.

This page is the raw list, not the analysis. For results per set with their limits, see the benchmark leaderboard. For how scoring works, see our methodology.