Bytedance-seedModel

Seed-2.0-Mini

How Seed-2.0-Mini performed across every Indonesian and regional-language benchmark we publish.Read more
Last runon Arti kata Jawa sehari-hariversion not pinnedKeterangan

Benchmarks tested

5

of 6 benchmarks we publish

The rest have not been run, so this page can say nothing about how it does on 1 other benchmarks.

Runs completed

8

complete runs on record

One run means the model answered every item in a set, start to finish. A set run more than once contributes more than one.

Total cost

$0.0989

for every run above

The real amount billed by the provider, summed, not an estimate from per-token pricing. Not yet divided by correct answers.

Total tokens

311,507

input and output combined

Counted only from runs that recorded tokens. Runs on the older path did not store them, and the figure is not guessed.

How many answers were correct

Each bar is one benchmark: how many items it answered correctly out of every item in that set. We wrote the questions and answer keys ourselves and had them checked by two native speakers, and every model answers exactly the same items.

8 benchmarks
Pemahaman slang Indonesia37/42 88.1%
Normalisasi teks Indonesiaquestion version 3Keterangan35/49 71.4%
Normalisasi teks Indonesiaquestion version 2Keterangan35/49 71.4%
Koreksi ejaan Indonesia22/38 57.9%
Normalisasi teks Indonesiaquestion version 1Keterangan25/49 51.0%
Tingkat tutur bahasa Jawamultiple choice13/27 48.1%
Arti kata Jawa sehari-hari29/135 21.5%
Tingkat tutur bahasa Jawaopen answer1/6 16.7%
Share of items answered correctly across 8 benchmarks

How far the cost pays off

Sorted by the cheapest correct answer first. The left column shows how many items were answered correctly, so a top row with a long left bar means both cheap and accurate.

8 with cost on record
Correct answersCost per correct answer
Normalisasi teks Indonesiaquestion version 2Keterangan71.4%$0.000169
Normalisasi teks Indonesiaquestion version 3Keterangan71.4%$0.000186
Pemahaman slang Indonesia88.1%$0.000203
Normalisasi teks Indonesiaquestion version 1Keterangan51.0%$0.000290
Koreksi ejaan Indonesia57.9%$0.000446
Arti kata Jawa sehari-hari21.5%$0.000515
Tingkat tutur bahasa Jawamultiple choice48.1%$0.001133
Tingkat tutur bahasa Jawaopen answer16.7%$0.003426
Correct answers against cost per correct answer

Tokens used

Input and output are kept apart because they are priced differently, often tenfold. A long output bar means the model talks a lot, and that is where nearly all the cost difference comes from.

85,528 in, 225,979 out
inputoutput
Pemahaman slang Indonesia12,826 · 23,707
Normalisasi teks Indonesiaquestion version 3Keterangan12,621 · 19,167
Normalisasi teks Indonesiaquestion version 2Keterangan13,380 · 17,370
Koreksi ejaan Indonesia12,948 · 34,634
Normalisasi teks Indonesiaquestion version 1Keterangan11,448 · 22,550
Tingkat tutur bahasa Jawamultiple choice6,220 · 55,086
Arti kata Jawa sehari-hari14,808 · 45,220
Tingkat tutur bahasa Jawaopen answer1,277 · 8,245
Tokens for one full run, per benchmark

Wait time per call

Median wait for a single call on each benchmark. Median rather than mean, so one stalled call does not move the number.

8 with wait time on record
Normalisasi teks Indonesiaquestion version 2Keterangan1.21 s
Normalisasi teks Indonesiaquestion version 3Keterangan1.62 s
Normalisasi teks Indonesiaquestion version 1Keterangan1.71 s
Pemahaman slang Indonesia1.75 s
Arti kata Jawa sehari-hari1.97 s
Koreksi ejaan Indonesia3.45 s
Tingkat tutur bahasa Jawamultiple choice7.89 s
Tingkat tutur bahasa Jawaopen answer8.12 s
Median wait for one call, per benchmark

Test coverage

This model has been tested on 5 of the 6 benchmarks we publish. The rest have not been run, so this page cannot say anything about how it does there.

5 of 6 benchmarks we publish
Log8 calls to this model, spread across 2 days.

This model is called through a provider id that may not pin a version, so the dates below state which weights were actually tested. Rows marked with a hyphen have a date we inferred from file history rather than recorded at call time, so we do not show a clock time for them.

6 runs
  1. 18:00Arti kata Jawa sehari-hariword meaning29/135
  2. 16:15Tingkat tutur bahasa Jawaopen answer1/6
  3. 16:12Tingkat tutur bahasa Jawamultiple choice13/27
  4. 02:12Normalisasi teks Indonesiaopen answerPrompt stating the key's rule35/49
  5. 01:57Normalisasi teks Indonesiaopen answerDetailed prompt forbidding word swaps35/49
  6. 01:46Normalisasi teks Indonesiaopen answerBuilt-in prompt25/49
2 runs
  1. 02:22Koreksi ejaan Indonesiaopen answer22/38
  2. 01:16Pemahaman slang Indonesiamultiple choice37/42
See the whole panel's run log →

← See all models