OpenAI's GPT-6 Astra Scores Below Its Own Cheaper Model on Rare Javanese Words

A flagship launched for reasoning, not vocabulary

OpenAI released GPT-6 Astra on 3 September 2026, positioning it as the flagship successor to GPT-5.6 Sol. The launch leaned on numbers built to startle: a reported 99.9% on ARC-AGI-3, up from 7.8% for Sol, and OpenAI president Greg Brockman's line that "it's not unreasonable to think we're now in the AGI era." That quote, more than the benchmark table behind it, is most of what went viral.

The 99.9% figure came from a harness OpenAI built itself. On ARC Prize's own neutral evaluation, the same model scored 62.7%, and ARC Prize's organizers were explicit that the number "does not claim this is AGI." Astra is also priced at $10 per million input tokens and $50 per million output tokens, 2.5 times GPT-5.6 Sol's promotional rate. None of that is about our test. We ran Astra through two of our own Javanese-language benchmarks, and the gap we found sits somewhere narrower and more mundane: recalling what an uncommon word means.

Losing to its own cheaper sibling

We asked GPT-6 Astra, Google's Gemini 3.5 Flash, and OpenAI's own GPT-5.6 Luna to define 43 Javanese words absent from NusaX, NusaWrites, and Javanese Wiktionary, our Rare Javanese word meanings benchmark. Every answer is checked against a speaker-reviewed key covering more than one accepted sense per word.

Astra answered 33 correctly, 76.7%. Gemini 3.5 Flash answered 40, 93.0%. GPT-5.6 Luna, the OpenAI model Astra is meant to replace, answered 35, 81.4%, 2 words ahead of its own successor.

The gap between Astra and Gemini clears statistical significance (z = 2.11, p ≈ 0.035). The gap between Astra and Luna does not (z = 0.53, p ≈ 0.60): at 43 items, 2 extra correct answers is not enough to call the older model reliably better, only enough to say the new one is not reliably better than it.

Running all 43 words through Astra cost $0.22, at OpenAI's list price. The identical run on Luna cost $0.004.

The run that cost 56 times less scored higher.

A tie where the task changed

The same models also sat our Javanese speech-level test, which checks whether a model picks the register a Javanese speaker would use with a given listener, a different skill from recalling a word's meaning.

The tie holds on the held-out slice too, the items never published so a later model cannot have trained on them: Astra and Gemini both answered 11 of 11 multiple-choice and 2 of 2 open items correctly there as well. Whatever separates Astra from Gemini on rare vocabulary, it is not a general reasoning gap. GPT-5.6 Luna, run on the multiple-choice half only, scored 85.2%, behind both.

Fewer tokens, not more speed

OpenAI and Artificial Analysis both describe Astra as roughly 70% more token-efficient than GPT-5.6 Sol, the model it replaced: a third of Sol's tokens on the Codex coding harness, a fifth of Claude Opus 5's, 65% fewer output tokens than Opus 5 on Agents' Last Exam. Every one of those figures measures the same thing, output tokens spent per task, not response time.

Our own logs let us check both at once, and the token-efficiency claim holds up. On the speech-level test, where Astra and Gemini 3.5 Flash tied exactly on accuracy, Astra used 16.3 times fewer output tokens per item on the multiple-choice half and 4.4 times fewer on the open-answer half. The same answer, for a fraction of the tokens.

Response time is a different story. Across all three comparisons, Astra was never the fastest model in the room. It was the slowest of the three on the rare-word test, the exact test it lost. It was also the slowest of the two models run on the open-answer half of the speech-level test, the one it tied for the best score on while using less than a quarter of Gemini's output tokens.

We could not find an independent report of Astra running slower than comparable models. That is not a contradiction of our numbers, it is an absence: Artificial Analysis's own provider page for Astra lists output speed, time to first token, and end-to-end response time as "No data available." The industry's standard place to check a model's raw latency has no figures for this one yet, so our own runs may be the first outside data point on the question at all.

One documented detail may explain part of the gap. Artificial Analysis notes that Astra's pricing includes a separate reasoning-token category, billed and timed alongside the words it actually returns. A model can spend real seconds thinking before it writes a short answer, and that thinking time will not show up in a token count advertised as efficient.

A narrow claim about a narrow test

43 words is a small sample, and every word on it was picked specifically because it is obscure. The finding here is not that GPT-6 Astra is a weak model. It matched the best score in the panel on a task requiring social judgment about who is speaking to whom. It is that on this one narrow, deliberately hard vocabulary list, OpenAI's newest and most expensive model did not outscore its own predecessor, and answered clearly fewer correctly than Google's cheaper Gemini 3.5 Flash.