Z-aiModel

GLM 5.3

How GLM 5.3 performed across every Indonesian and regional-language benchmark we publish.Read more
Last runon Arti kata Jawa langkaversion not pinnedKeterangan

Benchmarks tested

6

of 6 benchmarks we publish

This model has been run on every benchmark we publish.

Runs completed

9

complete runs on record

One run means the model answered every item in a set, start to finish. A set run more than once contributes more than one.

Total cost

$0.1783

for every run above

The real amount billed by the provider, summed, not an estimate from per-token pricing. Not yet divided by correct answers.

Total tokens

102,561

input and output combined

Counted only from runs that recorded tokens. Runs on the older path did not store them, and the figure is not guessed.

How many answers were correct

Each bar is one benchmark: how many items it answered correctly out of every item in that set. We wrote the questions and answer keys ourselves and had them checked by two native speakers, and every model answers exactly the same items.

9 benchmarks
Pemahaman slang Indonesia40/42 95.2%
Tingkat tutur bahasa Jawamultiple choice21/27 77.8%
Normalisasi teks Indonesiaquestion version 3Keterangan37/49 75.5%
Normalisasi teks Indonesiaquestion version 2Keterangan37/49 75.5%
Arti kata Jawa sehari-hari98/135 72.6%
Koreksi ejaan Indonesia26/38 68.4%
Normalisasi teks Indonesiaquestion version 1Keterangan30/49 61.2%
Tingkat tutur bahasa Jawaopen answer3/6 50.0%
Arti kata Jawa langka14/43 32.6%
Share of items answered correctly across 9 benchmarks

How far the cost pays off

Sorted by the cheapest correct answer first. The left column shows how many items were answered correctly, so a top row with a long left bar means both cheap and accurate.

9 with cost on record
Correct answersCost per correct answer
Tingkat tutur bahasa Jawamultiple choice77.8%$0.000283
Pemahaman slang Indonesia95.2%$0.000311
Normalisasi teks Indonesiaquestion version 2Keterangan75.5%$0.000345
Normalisasi teks Indonesiaquestion version 1Keterangan61.2%$0.000371
Arti kata Jawa sehari-hari72.6%$0.000391
Koreksi ejaan Indonesia68.4%$0.000499
Normalisasi teks Indonesiaquestion version 3Keterangan75.5%$0.000527
Tingkat tutur bahasa Jawaopen answer50.0%$0.000740
Arti kata Jawa langka32.6%$0.001075
Correct answers against cost per correct answer

Tokens used

Input and output are kept apart because they are priced differently, often tenfold. A long output bar means the model talks a lot, and that is where nearly all the cost difference comes from.

78,830 in, 23,731 out
inputoutput
Pemahaman slang Indonesia12,683 · 759
Tingkat tutur bahasa Jawamultiple choice5,692 · 464
Normalisasi teks Indonesiaquestion version 3Keterangan12,387 · 4,352
Normalisasi teks Indonesiaquestion version 2Keterangan13,002 · 1,558
Arti kata Jawa sehari-hari8,589 · 10,336
Koreksi ejaan Indonesia12,754 · 1,175
Normalisasi teks Indonesiaquestion version 1Keterangan10,640 · 1,485
Tingkat tutur bahasa Jawaopen answer1,219 · 581
Arti kata Jawa langka1,864 · 3,021
Tokens for one full run, per benchmark

Wait time per call

Median wait for a single call on each benchmark. Median rather than mean, so one stalled call does not move the number.

9 with wait time on record
Tingkat tutur bahasa Jawamultiple choice1.02 s
Koreksi ejaan Indonesia1.10 s
Normalisasi teks Indonesiaquestion version 1Keterangan1.16 s
Normalisasi teks Indonesiaquestion version 2Keterangan1.19 s
Pemahaman slang Indonesia1.21 s
Normalisasi teks Indonesiaquestion version 3Keterangan1.39 s
Tingkat tutur bahasa Jawaopen answer1.47 s
Arti kata Jawa langka1.66 s
Arti kata Jawa sehari-hari1.77 s
Median wait for one call, per benchmark

Test coverage

This model has been tested on all 6 benchmarks we publish.

6 of 6 benchmarks we publish
Log9 calls to this model, spread across 1 days.

This model is called through a provider id that may not pin a version, so the dates below state which weights were actually tested. Rows marked with a hyphen have a date we inferred from file history rather than recorded at call time, so we do not show a clock time for them.

9 runs
  1. 22:35Arti kata Jawa langkaword meaning14/43
  2. 22:15Arti kata Jawa sehari-hariword meaning98/135
  3. 21:45Normalisasi teks Indonesiaopen answerPrompt stating the key's rule37/49
  4. 21:38Normalisasi teks Indonesiaopen answerDetailed prompt forbidding word swaps37/49
  5. 21:31Normalisasi teks Indonesiaopen answerBuilt-in prompt30/49
  6. 18:44Koreksi ejaan Indonesiaopen answer26/38
  7. 08:30Tingkat tutur bahasa Jawaopen answer3/6
  8. 08:14Tingkat tutur bahasa Jawamultiple choice21/27
  9. 08:00Pemahaman slang Indonesiamultiple choice40/42
See the whole panel's run log →

← See all models