Invoice recognition accuracy: how to judge it

The question an accountant actually asks is not "does it use AI" but "how many invoices will I have to correct, and will I notice before it is too late?" It is the question discussed least often, because the honest answer depends less on the system than on what your documents look like.

Why "99% accuracy" tells you nothing

An accuracy percentage is almost always measured per value, not per document, and the difference matters. Suppose each invoice carries fifteen values you have to carry across correctly, and each one is recognised with 99% accuracy. Then roughly one invoice in seven still has at least one thing to fix. The number reads beautifully, and you are still reviewing every document.

The second problem is that the figure was measured on invoices the vendor chose for the measurement, not on invoices from your suppliers. It is not a lie — it simply is not an answer to your question.

What actually decides accuracy on your documents

• •whether invoices arrive as digital PDFs, or as scans and phone photos
• •how many suppliers you have, and how often they change the look of their invoice
• •whether several VAT rates are mixed on a single document
• •whether invoices run long — dozens of lines, annexes, monthly summaries
• •whether foreign suppliers are in the mix, where both the company details and the VAT treatment differ
• •whether handwritten quantities, stamps and overlapping print show up

A company whose invoices arrive by email as tidy PDFs from twenty regular suppliers, and a company where half the documents are receipts photographed by drivers, will get completely different results out of the same system. That is exactly why a single headline percentage does not help you decide.

How to measure it in one evening

You do not need a trial quarter or your whole archive. Last month and a couple of hours is enough.

• •take fifty invoices from last month — not the tidy ones, the ones that actually arrive
• •deliberately include the worst of them: a scan, a multi-line telecom invoice, a foreign supplier, a credit note
• •sort the results into three piles: the ones you did not touch, the ones that needed one field fixed, and the ones that were faster to type in by hand
• •measure time, not percentages: how long did all fifty take end to end

The third pile is the one that matters. If it is empty or nearly empty, the first two are only a question of speed. If the same supplier keeps landing in it, that is no longer an accuracy question but a rule question — and you solve it once rather than every month. And make sure you have something to compare against: how long the same fifty invoices take when they are typed in by hand. Without that number the result has no reference point, and the decision comes down to a hunch.

What must happen to the ones it gets wrong

This matters more than accuracy itself. An error you see in the review screen costs you half a minute. The same error, unnoticed on its way into the ledger, costs you an explanation a month later — or during an inspection.

• •amounts have to be reconciled against each other: if the lines do not add up to the header, you see it immediately, not after the export
• •a value recognised with any doubt has to be flagged, not quietly filled in with a guess
• •a correction has to be one action, not re-entering the whole document
• •a trail has to remain — who changed what, so nobody is guessing six months later

The accuracy that costs you time

If you still have to open every document and read it from the top, the time saved evaporates, whatever percentage is printed on the website. So the more practical thing to count is not recognition accuracy but how many documents pass without your intervention, and what one document costs.

What to realistically expect

The honest answer: tidy digital PDFs from regular suppliers go through with almost no intervention. A poor scan, a handwritten quantity and an invoice from a foreign supplier seen for the first time all need a look. A system should not pretend otherwise — it should show you which is which, so your review time goes where it is actually needed.