Indonesian Language Observatory

Benchmark Nusantara

We test AI models on Indonesian and its regional languages ourselves, using questions and answer keys we wrote and had checked by two native speakers. The numbers are published as they are, limits included.Read more
6 benchmarks471 itemsLast tested
Indonesia

Indonesian spelling correction

Accepting or rejecting spell-checker suggestions on real text.

54 items16 held back18 models
Score spread
Gemma 4 31B29/38 76.3%
18 Nov 2026See the results
Indonesia

Indonesian text normalisation

Turning text-message style writing into standard Indonesian.

69 items20 held back17 models
Score spread
open answer, range across 3 prompts17 models
MiMo-V2.6-Flash76%
18 Nov 2026See the results
Indonesia

Indonesian slang comprehension

Explaining what Indonesian slang words mean inside a sentence.

60 items18 held back21 models
Score spread
GPT-5.6 Luna42/42 100.0%
18 Nov 2026See the results
Jawa

Rare Javanese word meanings

Explaining 43 Javanese words absent from every open source.

43 items18 models
Score spread
Gemini 3.5 Flash40/43 93.0%
18 Nov 2026See the results
Jawa

Everyday Javanese word meanings

Explaining everyday Javanese word meanings verified by native speakers, part of it held back.

199 items64 held back20 models
Score spread
Gemini 3.5 Flash128/135 94.8%
18 Nov 2026See the results
Jawa

Javanese speech levels

Choosing the Javanese register that fits the person being addressed.

46 items13 held back22 models
Score spread

multiple choice

Gemini 3.5 Flash26/27 96.3%

open answer

Claude Sonnet 5.56/6 100.0%
18 Nov 2026See the results

The only regional language measured so far is Javanese. Others such as Sundanese, Madurese, Balinese, Batak, Buginese and Minangkabau are still being collected: the sets do not exist yet, so there are no figures. We publish numbers only for what has actually been tested.

Every figure here comes from the same questions for every model, and the key does not change mid-test.