Transparency

Benchmark run log

Every time we run an AI model against a benchmark set, the row is recorded here exactly as it happened. Not a rollup like the /benchmark leaderboards, but one row per call, so any summary figure on this site can be traced back to the actual run that produced it.Read more
134 test runs14 models tested6 benchmark setsMost recent run
GitHub

1 runs
  1. 03:19Ling-3.0-flashNormalisasi teks Indonesiaopen answerDetailed prompt forbidding word swaps32/49

1 runs
  1. 19:42Ling-3.0-flashTingkat tutur bahasa Jawamultiple choice18/27

13 runs
  1. 23:08Claude Haiku 4.5Pemahaman slang Indonesiamultiple choice38/42
  2. 23:04DeepSeek V4 Flash 0731Pemahaman slang Indonesiamultiple choice39/42
  3. 22:55Hunyuan A13B InstructPemahaman slang Indonesiamultiple choice29/42
  4. 22:49GPT-5.6 LunaPemahaman slang Indonesiamultiple choice42/42
  5. 22:43Gemma 4 31BPemahaman slang Indonesiamultiple choice41/42
  6. 22:39Ling-3.0-flashPemahaman slang Indonesiamultiple choice37/42
  7. 22:32Qwen3 30B A3B Instruct 2507Pemahaman slang Indonesiamultiple choice37/42
  8. 22:26Mistral Small 3Pemahaman slang Indonesiamultiple choice32/42
  9. 22:19Solar Pro 4Pemahaman slang Indonesiamultiple choice37/42
  10. 22:10Llama3 8B CPT Sahabat-AI v1 InstructPemahaman slang Indonesiamultiple choice40/42
  11. 22:06Gemma-SEA-LION v4.5 E2B-ITPemahaman slang Indonesiamultiple choice33/42
  12. 22:04Llama3 8B CPT Sahabat-AI v1 InstructPemahaman slang Indonesiamultiple choice40/42
  13. 22:01Llama 3.1 8B InstructPemahaman slang Indonesiamultiple choice32/42

14 runs
  1. 22:11Llama 3.1 8B InstructArti kata Jawa sehari-hariword meaning26/135
  2. 22:08Llama3 8B CPT Sahabat-AI v1 InstructArti kata Jawa sehari-hariword meaning61/135
  3. 22:05Gemma-SEA-LION v4.5 E2B-ITArti kata Jawa sehari-hariword meaning60/135
  4. 21:55Gemini 3.5 FlashArti kata Jawa sehari-hariword meaning128/135
  5. 21:44Nemotron 3.5 LightningArti kata Jawa sehari-hariword meaning25/135
  6. 21:32DeepSeek V4 Flash 0731Arti kata Jawa sehari-hariword meaning107/135
  7. 21:18Claude Haiku 4.5Arti kata Jawa sehari-hariword meaning86/135
  8. 21:10Ling-3.0-flashArti kata Jawa sehari-hariword meaning69/135
  9. 20:50GPT-5.6 LunaArti kata Jawa sehari-hariword meaning112/135
  10. 20:42Hunyuan A13B InstructArti kata Jawa sehari-hariword meaning21/135
  11. 20:35Gemma 4 31BArti kata Jawa sehari-hariword meaning116/135
  12. 20:12Mistral Small 3Arti kata Jawa sehari-hariword meaning25/135
  13. 20:06Solar Pro 4Arti kata Jawa sehari-hariword meaning52/135
  14. 19:57Qwen3 30B A3B Instruct 2507Arti kata Jawa sehari-hariword meaning51/135

1 runs
  1. 16:57Gemma-SEA-LION v4.5 E2B-ITTingkat tutur bahasa Jawaopen answer5/6

The number at the far right of each row is the score: correct answers out of items scored. It can be smaller than the set's full item count when some items are held back or still awaiting human judgment.

This page is the raw list, not the analysis. For results per set with their limits, see the benchmark leaderboard. For how scoring works, see our methodology.