The proof: an eval suite and a touchless-rate dashboard you keep.

During discovery week, we pull a sample of your actual invoice history, not a demo set and not a synthetic one. Typically a few hundred invoices spanning your highest-volume vendors, your messiest formats, and a deliberate sample of the edge cases that break naive automation: multi-page invoices, credit memos, invoices with no PO attached, near-duplicate resends, and foreign-currency invoices if your business has them.

Every invoice in that sample gets labeled by hand: every header field, every line item, the correct GL code, and the correct match verdict against the PO and receipt it should tie to. That labeled set becomes the golden dataset, the ground truth that every version of the extraction and matching logic gets tested against, during the build and for as long as you keep using the system afterward.

Because it is built from your invoices, it would be worthless to reuse for anyone else's build. That is the point. It tests what your system will actually see in production, not a generic benchmark that happens to look good in a sales deck.

The first check in the eval suite measures extraction against the golden dataset, field by field. Not "did it get the invoice roughly right," but did it get the vendor name right, the invoice number, the date, the PO reference, the subtotal, the tax, the total, and every line item, each one scored on its own.

Field-level scoring matters because a single transposed digit in a total is a different kind of failure than a slightly reformatted vendor name, and averaging them into one accuracy number hides the failure that actually costs money. We track precision and recall per field type, and we track it separately by vendor format, because a system that scores 99% on your cleanest vendor and 80% on your messiest one is not "95% accurate." It is two different systems wearing one average.

The second check is the one that matters most, and the one most automation vendors skip. It is straightforward to build a 3-way match that looks accurate in a demo. It is much harder to build one that never confidently approves a match that is actually wrong: a quantity off by a decimal, a price variance sitting just inside a badly-set tolerance, an invoice matched to the wrong PO because two vendors have similar names.

We call these confident-but-wrong matches, and they are the real risk in automated matching. A low-confidence match gets routed to a human by design, so it is caught. A high-confidence wrong match sails straight through, which is exactly why the regression suite exists: it runs known-bad pairs, near-miss pairs, and adversarial cases (deliberately similar vendor names, off-by-one quantities, stale price lists) through the matcher and checks that every one of them gets held, not waved through with high confidence.

This suite runs on every change to the matching logic, not once at launch and then forgotten. A tolerance adjustment that fixes one vendor's false exceptions cannot be allowed to quietly weaken the exception logic for every other vendor, and the regression suite is what catches that trade-off before it ships.

The third check is duplicate detection: the same invoice submitted twice, a legitimate credit memo that superficially resembles a duplicate, a resend after a vendor's billing system hiccups. Duplicate payment is one of the most common and most preventable losses in accounts payable, and it is also one of the easiest to catch with the right combination of signals, vendor, invoice number, amount, and date, checked consistently across a rolling window rather than assumed away by a spot check.

The eval suite includes known duplicate pairs and known near-duplicates that are not actually duplicates, because two invoices from the same vendor for the same amount in the same month are not automatically a repeat. We measure both the catch rate and the false-positive rate. A system that flags everything as a duplicate is exactly as useless to your AP team as one that misses them, it just moves the cost from fraud risk to review fatigue.

The fourth check is not a probability, it is a hard assertion. Every build ships with an automated test that scans the agent's complete action log for any attempt, successful or not, to initiate a payment, move funds, or write to a payment rail. That count has to be zero on every single test run, and it is verified before every deployment, not just once at handover.

This is not a policy the agent is asked to follow. It is a boundary enforced in code: the agent's available tools do not include a payment action, so there is nothing to attempt in the first place. The check is really a confirmation that the boundary held, not a hope that it will keep holding. Payment stays in your existing approval chain, exactly where your controls and your auditors already live.

The golden dataset and the eval suite prove the logic works against history. The parallel run proves it works against invoices arriving today. In week five, the agent processes your live invoice stream alongside your team, in parallel, not instead of them. Every invoice it captures, matches, codes, and either clears or routes to the exception queue gets compared against what your team actually did with the same invoice.

That comparison produces the number that matters most: your real touchless rate, measured on your live, unfiltered invoice stream, not a curated sample and not a vendor's benchmark measured on somebody else's invoices. It also produces something almost as useful, a breakdown of exactly which invoice types and which vendors are driving whatever that number turns out to be, which is usually the fastest way to see where your own upstream process has a gap worth fixing before you ever touch the model.

Cutover happens only after the parallel run clears the thresholds fixed in the contract. Until then, your team keeps doing exactly what they do today, and nothing changes for your vendors, your books, or your close.