Blind vote arena

Which AI actually writes Indonesian well?

Two AI models answer the same Indonesian question, be it slang, a regional dialect, or formal register, and you pick the better one. Model names are hidden until you vote, so what you judge is the writing, not the brand. Every answer was written and stored ahead of time, which is why the arena responds instantly.

categories
3
models in the arena
27
pairs to judge
10470

How a rank is produced

  1. 1

    You pick

    Two answers to the same question, names hidden and sides randomised by the server, so what you judge is the writing, not the brand, and a habit of picking one side never quietly becomes a result.

  2. 2

    Results are weighed

    A Bradley-Terry model turns head-to-head results into a strength estimate, so beating a strong opponent counts for more than beating a weak one.

  3. 3

    The gate holds

    Every model's rating is recomputed regularly as votes come in, but the number is not shown as a final ranking until a model has been judged enough times for it to mean something.

    See the ranking

Everything on the ranking page comes from this chain and nothing else. We do not fold in scores from a separate test: that measures whether a model knows the right answer to a closed item, which is a different question from which free-form answer a speaker accepts.

Questions people ask

How does the AI Battle work?

Two AI models answer the same Indonesian question, whether slang, a regional dialect, or formal register. You read both answers with the model names hidden, and pick the one that fits best. Only after your vote do we reveal which model wrote which answer, along with what each model's record in the arena looks like so far.

Why are the model names hidden until after the vote?

Because a visible name changes what is being judged. Shown the brand first, people rate the brand; shown only the writing, they rate the writing. Blind comparison is the only version of this that measures language quality rather than reputation. Which side an answer appears on is also decided by the server and randomised, so a habit of picking the left box does not quietly become a result.

Are the answers generated live when I open the page?

No. Every answer is written ahead of time and stored, then served as a pair. That is a deliberate choice: it means the arena responds instantly, the same answer can be compared against several opponents without being rewritten, and what you judge is a fixed text rather than a fresh roll of the dice.

How is the ranking scored?

Entirely from the votes cast here. A Bradley-Terry model turns head-to-head outcomes into a strength estimate for each model, which is not the same as counting wins: beating an opponent that rarely loses moves the estimate far more than beating one that usually does, so no model is rewarded for happening to face easy opponents. A rank is only published once a model has been judged often enough for its interval to be meaningful; below that, an order would be noise dressed as a result.

Why is a rank shown as a range rather than a number?

Because with a limited number of votes, several models cannot be told apart. Each estimate carries an interval, and where two intervals overlap there is no evidence that one model sits above the other. Printing a single position for both would be a claim the votes do not support. A model whose interval is not yet meaningful is given the full range, which is the correct answer to "what rank is it" when the answer is not known.

Why is there an "equally good" option?

Because forcing a choice between two answers that are both acceptable injects noise into the ranking. In categories judged on speaker acceptance rather than a prescriptive rule, two different answers being equally right is a normal situation, not an edge case.

Some pairs say the vote was not counted. Why?

A share of pairs are calibration items whose correct answer we already know. They measure how reliable the voting is overall, so they are deliberately excluded from the ranking itself. Votes cast in browsing mode, and votes returned too quickly to be a real reading, are also recorded but kept out of the ranking.

Does a longer or more elaborate answer win automatically?

That bias is real and well documented in this kind of arena, which is why we correct for it. Alongside the raw rating we estimate a style-adjusted one, which accounts for answer length and formatting. The gap between the two is itself a finding: it shows how much of a preference was rhetoric rather than language competence.

Glossary

Bradley-Terry
A statistical model that estimates each contestant's strength from head-to-head results. It does not simply count wins: beating an opponent that rarely loses moves the estimate far more than beating one that usually does, so a model is never rewarded for happening to face easy opponents.