Reconciliation is deterministic. AI belongs where judgment is.
reconciliationai-safetybankingarchitecturehuman-in-the-loop
Every few weeks an engineer posts a new take: “I used an LLM to reconcile two spreadsheets.” The model changes with the season and the file format changes with the customer, but the demo always looks impressive. In production — especially in banking — it’s a disaster waiting for the auditor.
Others have made this case well — Dharmendra Pratap Singh’s Stop Using LLMs to Compare CSVs is the clearest — and this post is how the argument lands inside a regulated product.
For the reader in a hurry. Reconciliation is arithmetic. A language model is a statistical guess machine. Ask it to match your transactions and you’ll get plausible-looking answers that are silently wrong — and no auditor will ever accept “the AI decided” as a reason. AI still earns its keep in three specific places: reading a smudged bank statement, spotting a payslip that’s been retouched, and explaining a case to a reviewer in plain English. The right architecture is a deterministic core with AI at the edges — and a human on every binding decision. This is how Recon is built.
For engineers. The rest of this post covers the engineering choices: greedy 1:1 matching with configurable tolerances (borrowed from two open-source reconcilers), specialist agents that produce evidence rather than verdicts, an OCR salvage ladder that only escalates when Document AI’s confidence drops, and an audit trail that captures every LLM call alongside the human decision it informed.
A bank sends a customer a statement. The customer’s ledger says something different. Someone has to figure out which side is wrong before payroll runs, before the loan gets funded, before the auditor comes back with follow-up questions.
Reconciliation is the friction that eats loan-officer afternoons, delays credit decisions, and — when it goes wrong — sits at the centre of the enforcement action nobody wants their name attached to. It’s not glamorous. It is where trust in a financial institution is quietly built or broken every day.
Almost every institution still runs this on a stack of SQL scripts, Excel lookups, and manual eyeballing. When numbers don’t match, someone has to answer:
The moment a team sees a modern language model, the temptation is obvious: dump both files into the prompt, ask for the differences, ship. In a five-row demo it looks like magic. Then the problems start.
Language models don’t do arithmetic. They predict tokens. Ask any of them to reconcile 148 transactions and it will confidently produce an answer that is plausible-sounding, undated, and off by several rows. Reconciliation is deterministic — the same two spreadsheets should produce the same result today, tomorrow, and in the second-line-of-defence review next quarter. A stochastic model can’t promise that.
Scale breaks the illusion fast. A branch’s daily transactions blow past any useful context window. Bank statements often run to hundreds of pages once you include prior-period comparatives.
Cost compounds. Every reconciliation is a fresh, expensive inference. Deterministic code runs at essentially zero marginal cost.
Auditability collapses. Regulators want reproducibility. They want to know which rows matched, on what basis, and why the ones that didn’t were classified the way they were. “The AI decided” is not an answer that survives a supervisory review. It is not an answer that survives an internal audit, either.
None of this is a takedown of language models. It’s a takedown of using the wrong tool for the wrong job.
If a computer can do it deterministically, we don’t ask a language model to do it.
Everything else in Recon follows from that one sentence.
Under the hood, a case looks like this:
┌──────────────────────────────────────────────────────┐
│ 1 Documents in (statements, ledgers, applications) │
└───────────────────────┬──────────────────────────────┘
│
┌───────────────────────▼──────────────────────────────┐
│ 2 Ingest + extract (Document AI, OCR fallback) │
└───────────────────────┬──────────────────────────────┘
│
┌───────────────────────▼──────────────────────────────┐
│ 3 Deterministic reconciliation engine (TypeScript) │
│ greedy 1:1 matching · date window · amount ε │
└───────────────────────┬──────────────────────────────┘
│
┌───────────────────────▼──────────────────────────────┐
│ 4 Specialist agents produce evidence │
│ arithmetic · provenance · cross-doc consistency │
└───────────────────────┬──────────────────────────────┘
│
┌───────────────────────▼──────────────────────────────┐
│ 5 Human makes the decision │
│ recorded in the audit log with the evidence │
└──────────────────────────────────────────────────────┘
The reconciliation engine in step 3 is where the deterministic-first rule bites hardest. It’s a pure TypeScript module (lib/reconcile/match.ts) synthesised from two open-source reconcilers we studied: oprekable/bank-reconcile, which enforces 1:1 matching with a SQL window function, and Abstra’s template, which layers tolerances on top for the ambiguous tail. Our version does the equivalent greedy 1:1 assignment in memory. It has no database access, no side effects, no LLM anywhere in its call graph. That means you can unit-test it in milliseconds and — critically — a second reconciler can run the same inputs and get identical outputs. That is what “reproducible” means in a compliance conversation.
// Real shape from lib/reconcile/match.ts — trimmed for the blog.
export interface MatchTxn {
id: string;
date: string; // ISO YYYY-MM-DD
amount: number;
type: string; // "credit" | "debit"
currency: string;
description: string;
documentId: string;
}
export interface MatchOptions {
/** Max absolute date difference, in days, still considered a match. 0 = exact. */
dateToleranceDays?: number;
/** Max absolute amount difference still considered a match. 0 = exact. */
amountTolerance?: number;
/** If true, a credit may match a debit (rare; default false — type must match). */
ignoreType?: boolean;
}
export interface MatchResult {
matched: MatchedPair[]; // exact + fuzzy_date + fuzzy_amount + fuzzy_both
unmatchedSource: MatchTxn[];
unmatchedTarget: MatchTxn[];
summary: { /* counts, deltas, kind histogram */ };
}
export function matchTransactions(
source: MatchTxn[],
target: MatchTxn[],
options?: MatchOptions,
): MatchResult { /* … */ }
That signature is the promise. Same inputs, same result. Every time.
If deterministic code is the load-bearing wall, AI is the finish work. Here’s where it actually adds value in a reconciliation product:
Getting numbers out of the messy documents in the first place. Bank statements arrive as PDFs of varying quality. Payslips come from a hundred different payroll systems. Application forms show up as scans, sometimes rotated, sometimes with a coffee ring on them. Recon runs each document through Google Document AI first (structured, high-confidence extraction). When Document AI’s confidence drops below a configurable threshold — 0.80 in our defaults — we escalate to a multimodal language model to salvage identity and account fields before flagging the document unreadable. That’s a salvage ladder, not a substitute. The deterministic system is doing the work; the LLM catches what the deterministic system genuinely can’t.
Reading intent across documents in the same case. A payslip in a loan application file says the account holder is A. Diallo. The bank statement in the same folder says Amadou Diallo. A regex-based cross-check would flag those as a mismatch and stop; a human reviewer would sigh, pattern-match, and move on. Our cross-check tool uses the LLM to decide whether that’s the same person — but it produces a signal (match / partial / mismatch) that a human reviewer sees and can override, not a decision on the case.
Document forensics that a computer can actually do. PDFs carry structural evidence of tampering that a language model reads as easily as we do: producer strings, incremental-update revision counts, ModDate earlier than CreationDate, embedded annotations that don’t belong on a bank-issued statement, image-only PDFs pretending to be text. Recon’s Tier 1 provenance check is deterministic byte-level analysis (lib/forensics/provenance.ts). Tier 2 is a multimodal LLM triage — explicitly framed as low-confidence, one signal among many. Tier 3 (pixel-level ML forensics, ELA, resampling detection) is deliberately deferred until real fraud volume justifies buying a specialist vendor.
Explaining what happened in plain English. After the deterministic engine has done its work, an agent writes a short summary — “142 of 148 transactions matched. The six unmatched entries on the customer side all fall in the last four days and are small credits; consistent with a settlement lag.” A reviewer scans that in five seconds, drills into the specific rows if it’s interesting, and moves on. This is where language models are genuinely superhuman. It’s also the only place in the pipeline where their output is a summary rather than a decision.
The bright line: no agent tool in Recon can approve, reject, or otherwise finalise a case. The reconciliation agent produces a match ledger. The document-validation agent produces evidence — pass, review, flag. The principal orchestrator collects that evidence and hands it to a human sitting in /cases/[id] with buttons that say approve, reject, escalate. The human clicks one. The click is written to an append-only audit log with the decision-maker’s identity, the case ID, and the evidence that was on their screen when they made it.
We took that stance for two reasons, in order of importance.
Responsible-lending law forbids automated decisions in most credit contexts. In Australia this is the responsible-lending obligations under the National Consumer Credit Protection Act. In the EU it is GDPR Article 22, which restricts decisions based solely on automated processing that produce legal or similarly significant effects on the individual — and requires a route to meaningful human review. Any product that lets an AI approve a loan is a product that assumes the operator can defend that architecture in a regulator’s office. Most of our customers cannot, and neither can we.
Stochastic decisions are indefensible even when they’re right. An audit doesn’t just ask “was the decision correct?” — it asks “was the process the same one that would have been applied if the customer was different, or the reviewer was different, or the day was different?” A deterministic pipeline + a documented human decision passes that test. An LLM verdict — even a good one — does not.
Recon writes an audit row for every action that could later be interesting to a regulator or a support team: case created, document uploaded, agent run started, agent tool call, LLM prompt and response, human decision, user account deleted. Structured JSON, separate from application logs. There is a corresponding AuditLog table in Postgres, and the code path is lib/logger.ts.
Concretely: if an applicant disputes a decision six months later, we can reconstruct exactly what evidence the reviewer had on screen, which model produced which explanation, and which button they clicked. Every LLM prompt that ever ran on their file is retrievable.
That is what “AI in a regulated environment” looks like when you take the regulation seriously.
prevent_destroy on production surfaces. The boring correctness that keeps a bank-grade product in service.None of this is exciting to look at. All of it is the reason a compliance team can sign off on the tool.
Do the arithmetic with code. Ask the language model to explain what the code found. Ask a person to make the call. Write everything down.
That is what a reconciliation product looks like when it’s built for the world where it will actually be used.
Recon is Nasugn’s AI reconciliation and document-verification product for banks, lenders, and payments companies. If this is the shape of a reconciliation tool you’d want to run, request a demo — a human replies, usually the same day.