BenchmarkEmpirical
We asked 10 AI models the meaning of 135 Javanese words. Gemini 3.5 Flash answered the most correctly, 128 out of 135, and is at the same time the worst value of the lot: 187 times more expensive per correct answer than a model that trails it by just 11 words.
September 22, 2026 · 4 menit bacaRead the finding →
BenchmarkEmpirical
We put 4 AI models through the same 69 sentences using three ways of asking. Gemma 4 31B, built by Google, came last under the first way and first under the third, with the questions and the answer key never changing.
September 15, 2026 · 4 menit bacaRead the finding →
BenchmarkEmpirical
On August 24 we wrote that ox-alpha was not GLM-5.3 but a relative of it. On August 26, Z.ai announced the model was GLM-5.3-Flash. What took us there was not a hunch, but 38 Javanese questions.
August 30, 2026 · 3 menit bacaRead the finding →
Correction engineStructured Data
Before writing grammar rules from scratch, we first checked what could be borrowed from LanguageTool, the largest open-source grammar checker there is. It has modules for 35 languages. The world's fourth most spoken language isn't one of them.
August 21, 2026 · 2 menit bacaRead the finding →
BenchmarkEmpirical
Claude Haiku, Anthropic's cheapest model, answered 10 of 36 Javanese politeness-level questions wrong. Claude Sonnet, its expensive counterpart, missed only 3.
August 19, 2026 · 3 menit bacaRead the finding →
Correction engineEmpirical
We pointed our own spell checker at real writing people produced, not test sentences. It flagged 3,741 things, and most weren't errors. This is the record of six rounds of fixing it, including a number we miscounted ourselves.
August 17, 2026 · 3 menit bacaRead the finding →
CorpusEmpirical
Wikipedia's edit history looks like a ready-made source of spelling corrections. Of the 744 we mined, only 43 were genuinely fit to become test items. The rest weren't real corrections.
August 15, 2026 · 2 menit bacaRead the finding →
JavaneseEmpirical
We built Javanese test items from grammar-book rules. Six of 40 items were dropped, not because AI answered wrong, but because the form the book called wrong turned out to be widely accepted by native speakers.
August 14, 2026 · 3 menit bacaRead the finding →
DictionaryStructured Data
We looked for words Indonesians use often that are absent from our dictionary. We found 3,289 candidate new words, but once checked closely, only 899 genuinely qualified. The rest were brand names, place names, or words barely used at all.
August 13, 2026 · 3 menit bacaRead the finding →