Accuracy
Measured from reviewer corrections as people work, not sampled quarterly.
Field accuracy
Confirmed over confirmed plus corrected, for fields a person actually looked at. What the model gets right when it was asked.
| Field | Confirmed | Corrected | Accuracy |
|---|---|---|---|
| Invoice number | 1 | 0 | 100.0% |
| Issue date | 4 | 0 | 100.0% |
| Due date | 1 | 0 | 100.0% |
| Supplier name | 2 | 1 | 66.7% |
| Supplier tax ID | 2 | 0 | 100.0% |
| Currency | 1 | 0 | 100.0% |
| Subtotal | 1 | 0 | 100.0% |
| Total | 2 | 0 | 100.0% |
Auto accept precision
The same ratio over the random sample of fields that were accepted without review. This is the number that answers whether the fields nobody checked were safe to not check.
| Precision | 93.3% |
|---|---|
| Sample size | 15 |
| Corrected in sample | 1 |
| Corrected outside the sample | 3 |
3 auto accepted fields turned out to be wrong when someone looked. The threshold is letting errors through.
Time saved
Fields a person never had to touch, times the manual baseline. Only fields nobody looked at count, which is the strictest honest reading.
| Time saved | 19m 0s |
|---|---|
| Fields never touched | 76 |
| Manual baseline | 15s per field |
Settings these numbers were judged by
Recorded on each extraction at the time it ran, so tuning them later does not rewrite what already counted as auto accepted.
| Threshold | Sample rate |
|---|---|
| 0.85 | 0.1 |
Every figure above traces to one query in docs/architecture.md.