Insight

How far can AI and our own language tools be trusted?

We test our own correction engine and various outside AI models against Indonesian and regional languages, then publish the numbers as they are. On this page, we sum up each finding in a single picture. The full method and raw data are one click away on every card.

4 findingsEvery finding links to its technical version
FilteredBenchmarkClear filter

4 findings

BenchmarkEmpirical
Cost rank Score rank
Gemma 4 31B
1
3
Qwen3-30B-A3B
2
7
Ling 3.0 Flash
3
6
GPT-5.6 Luna
4
2
DeepSeek V4 Flash
5
4
Mistral Small 3
6
9
Hunyuan A13B
7
10
Claude Haiku 4.5
8
5
Nemotron 3.5 Lightning
9
8
Gemini 3.5 Flash
10
1
Cost rank against score rank, 1 is best

The Smartest AI on Javanese Words Is 187 Times More Wasteful

We asked 10 AI models the meaning of 135 Javanese words. Gemini 3.5 Flash answered the most correctly, 128 out of 135, and is at the same time the worst value of the lot: 187 times more expensive per correct answer than a model that trails it by just 11 words.

September 22, 2026 · 4 menit bacaRead the finding →
BenchmarkEmpirical
Wording 1 Wording 3
Gemma 4 31B
32
60
Qwen3-30B-A3B
40
46
Ling 3.0 Flash
39
41
Mistral Small 3
39
38
Correct answers out of the exact same 69 sentences

Change the Prompt, and the AI Ranking Flips

We put 4 AI models through the same 69 sentences using three ways of asking. Gemma 4 31B, built by Google, came last under the first way and first under the third, with the questions and the answer key never changing.

September 15, 2026 · 4 menit bacaRead the finding →
BenchmarkEmpirical
11
Different 11Same 27
Answers from ox-alpha and GLM-5.3 on the same 38 questions

Two days before Z.ai spoke up, our data already said this was not GLM-5.3

On August 24 we wrote that ox-alpha was not GLM-5.3 but a relative of it. On August 26, Z.ai announced the model was GLM-5.3-Flash. What took us there was not a hunch, but 38 Javanese questions.

August 30, 2026 · 3 menit bacaRead the finding →
BenchmarkEmpirical
Claude Haiku 4.5
72.22
Claude Sonnet 4.6
91.67
Correct answers out of 36 Javanese politeness-level questions

Tested on Javanese Speech Levels, Anthropic's Cheapest AI Got It Wrong 10 Times

Claude Haiku, Anthropic's cheapest model, answered 10 of 36 Javanese politeness-level questions wrong. Claude Sonnet, its expensive counterpart, missed only 3.

August 19, 2026 · 3 menit bacaRead the finding →