Insight

How far can AI and our own language tools be trusted?

We test our own correction engine and various outside AI models against Indonesian and regional languages, then publish the numbers as they are. On this page, we sum up each finding in a single picture. The full method and raw data are one click away on every card.

3 findingsEvery finding links to its technical version
FilteredOur own failuresClear filter

3 findings

Correction engineEmpirical
With AI Without AI
Correct (overall)
77.1
88.3
Swapped letters
63.4
91.4
Deleted letters
71
79
Substitution
82
89
Doubled letters
91
94
Suggestion accuracy, with AI vs without AI

We Added AI to Our Spell Checker and Accuracy Dropped 11 Points

We added a language model to pick the best spelling correction from a sentence's meaning. On 1,500 deliberately corrupted words with a known answer, suggestion accuracy fell from 88.3% to 77.1%, and the engine became 67 times slower.

November 18, 2026 · 4 menit bacaRead the finding →
BenchmarkEmpirical
Wording 1 Wording 3
Gemma 4 31B
32
60
Qwen3-30B-A3B
40
46
Ling 3.0 Flash
39
41
Mistral Small 3
39
38
Correct answers out of the exact same 69 sentences

Change the Prompt, and the AI Ranking Flips

We put 4 AI models through the same 69 sentences using three ways of asking. Gemma 4 31B, built by Google, came last under the first way and first under the third, with the questions and the answer key never changing.

September 15, 2026 · 4 menit bacaRead the finding →
Correction engineEmpirical
3,741 Round 1 2,729 Round 6
Number of engine flags, round to round

Passed the Exam, Failed on Real Writing

We pointed our own spell checker at real writing people produced, not test sentences. It flagged 3,741 things, and most weren't errors. This is the record of six rounds of fixing it, including a number we miscounted ourselves.

August 17, 2026 · 3 menit bacaRead the finding →