Our correction engine failed on real text, then dropped 27% in two days

Language correction tools are almost always evaluated on sentences built to evaluate them. Short sentences, one error each, clean context. Ours passed there, and for a while we felt reasonably good about it.

Then we pointed it at writing people had actually produced.

Round one: 3,741 flags, and most of them wrong

The first number was 3,741. Not 3,741 errors, but 3,741 things the engine thought needed fixing. Once we went through them, most turned out not to be errors at all.

Place names treated as misspellings. Conversational particles glued onto the word in front of them. Technical terms "corrected" into words nobody meant. In one case the engine confidently turned the name of a district into an adjective.

A correction tool that is wrong more often than right is not merely useless. It is harmful, because people tend to accept a suggestion that appears on screen, especially one that looks confident.

How we measured it, and one choice worth challenging

We ran the engine over the same corpus repeatedly, saving the output to a separate file each round. Eight files. Two of them later turned out to hold identical content, and one more is the undated file that keeps being overwritten, so the genuinely distinct and frozen count is six.

Why files and not a database: a local database can be reset, and nobody outside the machine that ran it can audit it. Saved files can be compared pairwise, and that is what makes the change between rounds visible.

The challengeable choice: we count flags raised, not the share of flags that were correct. That share is far more meaningful, but computing it requires a person to inspect thousands of cases every round. We picked the measure we could run hourly, and accepted that it is blunter.

The numbers moved, but not the way we expected

Down 27% from the first round. Every drop has its own cause, and not one of them was raising the confidence threshold.

We deliberately avoided that route. Raising the threshold does lower the number quickly, but it hides the problem instead of fixing it, and the first things to disappear are the flags that were right. A good number produced by the engine doing less is not progress.

The most surprising part came from the biggest fix. When one rule stopped gluing particles onto the preceding word, those particle tokens did not vanish. They fell through to the next rule, which had no data for them, and guessed. So the total dropped while the kind of error changed.

The fix that lowered the count the most turned out to move the problem rather than remove it.

None of that would be visible from a single final snapshot. Two consecutive snapshots surfaced it in minutes.

The numbers in the first version were each one short, and here is why

The series above has been corrected. The first version carried seven numbers, and not one of them was right.

The first six were each short by one. The cause was wc -l, which counts newlines rather than records. Our output files do not end with a newline, so every count lost its last record.

wc -l reported 3,740. The file holds 3,741 records. Its final byte is }, not a newline.

The seventh number, 2,554, is a different and larger problem: it is in no file at all. It was read from the undated output file that gets overwritten every time the audit runs, so its contents have changed several times since this piece first went out. That number was removed rather than corrected, because there is no frozen state left to point at.

The drop is therefore 27%, not 32%.

The six remaining numbers come from dated snapshots that no longer change, and each can be checked against the snapshot summary in the raw data links below.

An irony we were not looking for: this piece argues that saved files are more auditable than a database, and then took one of its own numbers from a file that was indeed saved but never frozen. Saving is not enough. What makes a number checkable is its date and its content fingerprint.

What this does not show

The 27% figure is a drop in flags raised, not a gain in accuracy. The two correlate, but they are not the same thing. An engine that stops flagging anything scores a 100% drop and is completely useless.

We also did not compare against other tools. Not because the result would embarrass us, but because we have not run that comparison. Writing it up without running it is exactly the kind of claim that would make a piece like this worthless.

And two days is a very short window. The pace of repair over two intensive working days says nothing about the pace over the next month. If the curve flattens, we will write that up too.