Your confidence threshold is set by error cost, not savings

A vendor demo processes 94% of invoices without a human touch. The same model on your shared mailbox does 51%. Nobody cheated: the demo ran on clean digital PDFs from twelve suppliers, and your mailbox has three hundred and forty, about forty of whom photograph a delivery note on a phone.

Once the model is chosen, the confidence threshold is the only lever left — and almost every discussion of it happens without the arithmetic.

The review queue is cheaper than the business case assumes

Take 4,000 invoices a month. At a threshold that clears 65% straight through, 1,400 documents land in a human queue. A well-built review — source image cropped to the disputed field, extracted value pre-filled, failed rule named — takes 40 to 90 seconds. Call it 29 hours a month.

Eurostat put Bulgaria’s 2025 hourly labour cost at EUR 12.0 against an EU average of EUR 34.9; fully loaded, call it EUR 20 an hour in Sofia, so the queue costs about EUR 580 a month. Lower the threshold until 77% clears and you remove roughly 480 reviews: ten hours, about EUR 200 a month. That is the entire prize on the labour side of the decision, and it is smaller than one wrong payment. The threshold is set by what an escaped error costs, not by what a saved review earns.

The average hides the shape. The queue is bimodal: most items are a sixty-second correction, but ten to twenty percent are chases — no purchase order, a goods receipt that never posted, a supplier whose bank details changed — running 6 to 15 minutes through somebody else’s inbox. Two hundred chases at ten minutes is 33 hours, more than the whole rest of the queue. Report both populations separately, because only the first responds to a threshold. No threshold fixes maverick buying.

The expensive failures sit at the boundary, not among the obvious rejects

An upside-down scan, a blank cover page, a delivery note classified as an invoice — the reviewer disposes of these in seconds and nothing bad happens. Obvious rejects are cheap. What costs money is the confidently wrong field: a total read as 1100.00 where the document says 1.100,00, because the decimal comma was taken for a thousands separator; the previous-balance line picked instead of the amount due; a supplier’s order reference lifted into the invoice-number field, quietly disabling the duplicate check. All three come back around 0.97 and post unseen.

Field confidence describes the model’s own decoding — how sure it is that those glyphs are those characters. It does not know your supplier prints two totals. So check calibration before setting a cutoff: bucket a labelled sample by reported confidence and measure actual field accuracy inside each bucket. If documents scoring 0.95 are wrong 6% of the time, you have a ranking score — fine for ordering a queue, useless as an absolute cutoff. Derive the cutoff from the measured curve on your own documents, and re-measure after every model version, because the curve moves and nobody tells you.

Three arithmetic identities outrank any confidence score

Deterministic validation is free, has no variance, carries no per-page charge and does not drift when the vendor ships a new model. Confidence has all four problems.

An invoice’s net, VAT and gross are not three independent extractions; they are one claim with two degrees of freedom. If net plus VAT does not equal gross, at least one field is wrong whatever the model reported. You do not learn which — but you have learned, for nothing, that this document must not post. Line items repeat that trick twelve times over: quantity times unit price must equal the extended amount, and the extended amounts must sum to the net. Three fields at 0.98 each become one testable claim.

IBANs carry a mod-97 check under ISO 13616 and ISO 7064 — move the first four characters to the end, map letters to digits A=10 through Z=35, and the remainder on division by 97 must be 1. Bulgarian IBANs are 22 characters. That one check catches essentially every single-character OCR error and every transposition in the most dangerous field on the page. VAT identifiers get a structural check plus a live VIES lookup; a Bulgarian EIK carries its own weighted-modulus check digit. An IBAN that validates but differs from the last-known-good for that supplier is not a confidence problem at all — it is the most exploited fraud vector in European back offices, and belongs in a dual-approval workflow with an out-of-band callback, never in the extraction queue.

Run the deterministic layer first and let confidence decide only the residue: the fields no rule constrains, a free-text description, a delivery address. Most routing decisions you assumed needed a threshold turn out to be checksum failures.

A per-document gate collapses under multiplication

Published field-level F1 runs 95-99% on header fields — invoice number, date, total, VAT identifier — on clean digital PDFs, and 75-90% on line items, lower on degraded Cyrillic scans. A twelve-line invoice with four fields per row is 48 extractions. At 90% each, the probability that all 48 are simultaneously right is 0.9 to the 48th: about 0.6%.

So a document-level accuracy figure is not a metric, and gating the document on its weakest field sends everything to review. Route per field: post the header when the header clears and the arithmetic foots, and send only the disputed row to a reviewer with that row highlighted. And where supplier, total and VAT are right and line detail only feeds cost allocation, ask whether you need line-level extraction at all — for many non-stock invoices you do not, and dropping it buys more straight-through processing than any model change will.

An STP rate quoted without a document mix describes someone else’s mail

Manual accounts-payable functions run 15-35% touchless at baseline. Published achievable ranges are 65-85% on PO-backed invoices and 30-50% on non-PO. If 40% of your volume is non-PO, the blended ceiling is around 0.6 × 0.75 + 0.4 × 0.40 — about 61%, before anyone has looked at scan quality.

Then stratify by input. Amazon Textract detects text in English, French, German, Italian, Portuguese and Spanish only, handwriting in English only — no Bulgarian, no Cyrillic — and its minimum detectable text height is 15 pixels, roughly 8pt at 150 DPI. That fact alone removes it from most Bulgarian back-office pipelines before the accuracy conversation starts. Azure AI Document Intelligence and ABBYY do handle Cyrillic; Azure’s S0 tier defaults to 15 analyse transactions per second, with a documented 2-5-13-34 second backoff on 429s.

So the only accuracy commitment worth signing is per field, on your own sample, stratified by document class — digital PDF, clean scan, phone photograph, Cyrillic scan — with the class mix stated as part of the number. Roughly 300 documents per class pins a 95% field accuracy to about ±2.5 points, so do not contract on a difference smaller than that. And contract the residual exception rate, not only the automation rate: expect 15-30% residual exceptions in year one on a real supplier mix. Vendors quote the numerator.

A correction that is not written back is a treadmill

Every review should leave something behind: a supplier layout hint, an article cross-reference, a last-known-good IBAN, a tolerance rule that should have existed. Otherwise the same document returns next month at the same confidence. And publish the age of the oldest queue item beside the straight-through rate: work moved into an ageing queue was relocated, not automated.

Where this stops applying

There is a volume floor. At 300 documents a month the queue costs on the order of EUR 1,440 a year on a Bulgarian cost base; no routing logic pays that back, so the justification has to be cycle time, duplicate-payment control or audit evidence — or there is none. Routing also assumes a reviewer who can tell right from wrong in seconds; where the correct value needs domain judgment they do not have, routing relocates the error and adds delay to it.

Draw the legal boundary precisely rather than letting it stop the build. Matching an invoice to a purchase order is not a decision about a person and carries none of GDPR Art. 22. A pipeline that auto-rejects someone’s claim or application does, and then owes human intervention, the right to express a view and the right to contest.

Last, about us. Palamed has not run a production document-extraction pipeline. Our four deliverable engagements are a European car marketplace with more than 300,000 listings, a platform for an AI automation agency, the Ministry of Education and Science dictionary at beron.mon.bg, and email-marketing automation for a beauty brand. Everything above is the published pattern, the vendors’ own documentation and arithmetic you can check yourself. What we would measure first on your mail is the document-class mix and the calibration curve, before anyone names a threshold. If a supplier quotes a straight-through rate before seeing your documents, they have quoted someone else’s.