Where rules-only matching stops

Every reconciliation system starts the same way. Match on reference and amount. It works. Then the exceptions arrive, each one producing a new rule, until the rule set is four hundred lines long, nobody remembers why rule 217 exists, and the match rate has been stuck at 92% for a year.

The plateau is not a failure of effort. Rules are the right tool for a specific shape of problem, and the last 8% is a different shape.

What rules are good at

A rule is a predicate over a pair of records. It is exact, fast, auditable and trivially explainable — the reference matched, the amount matched, done. For one-to-one relationships where both sides carry a shared key, nothing beats it, and nothing should replace it.

The trouble starts when the relationship is not one-to-one, or when the shared key is damaged.

Problem one: the key is damaged

A payout instructed with reference PO-4482-KESTREL arrives on the statement as PO4482KESTR, because a bank in the chain truncated the field to eleven characters. Amount, currency and direction all agree. A human matches this in under a second. A rule cannot, because equality is the only comparison it has.

The instinct is to add a rule for truncation. Then one for banks that uppercase and strip hyphens. Then one for a sender that prefixes its own batch id. Each rule is correct and each one narrows the problem slightly, and the tail never ends because there is no finite list of ways a string can be damaged.

What works is a similarity measure with a threshold, applied only when the amount already agrees exactly. Trigram similarity over a sliding window handles truncation, reformatting and transposition in one mechanism.

// Verbatim containment wins outright. Otherwise slide a
// needle-sized window and keep the best local score, so a
// short reference is not drowned by a long bank narrative.
function refScore(needle: string, haystack: string): number {
  if (haystack.includes(needle)) return 1;
  let best = 0;
  for (let i = 0; i + needle.length <= haystack.length; i++) {
    best = Math.max(best, dice(needle, haystack.substr(i, needle.length)));
  }
  return best;
}

Note what this does not do. It does not decide anything on its own. The amount must already be identical, the direction must agree, and the value date must be inside the window. Similarity is the corroborating evidence, never the primary one — which is why the confidence it produces can be reported honestly and a reviewer can disagree with it.

Problem two: the relationship is not one-to-one

Five payouts are submitted in one pain.001 file. The bank debits the account once. Your ledger has five rows and the statement has one, and no predicate over a pair of records can express the relationship, because the relationship is not between a pair.

This is a subset-sum problem: find the subset of unmatched instructions whose total equals the debit. It is NP-complete in general, which sounds fatal and is not, because the instances that occur in reconciliation are heavily constrained.

• Candidates share a currency, a direction and a narrow date window — usually fewer than forty records.
• No individual instruction can exceed the batch total, which prunes aggressively.
• Real batches are small. Bounding the search at eight items covers the overwhelming majority and caps the cost.
• A suffix-sum table prunes any branch whose remaining candidates cannot reach the target.

With those constraints a depth-first search with pruning finds the subset in well under a millisecond. The genuinely hard instances — hundreds of same-value candidates — are also the ones where any answer would be ambiguous, so the correct behaviour is to decline rather than to guess. An engine that returns nothing is recoverable. An engine that returns a plausible wrong batch is not.

Problem three: the amounts are supposed to differ

A card capture is booked gross. The acquirer settles net of merchant discount. The amounts will never be equal and should never be equal, so equality-based matching cannot work at all — and a naive tolerance is worse, because a 3% tolerance on every comparison will happily match two unrelated transactions that happen to sit within 3% of each other.

The fix is that the tolerance must be directional and contractual. The settlement may be lower than the capture, never higher. The gap must fall inside the band in the acquirer agreement, computed per transaction. And the residual is not absorbed — it is posted to processing costs, which means it is visible, and a fee that drifts outside the band becomes a break rather than a rounding difference.

Where the model belongs

Nothing above involves a language model, and that is deliberate. Similarity scoring, subset-sum and tolerance bands are deterministic algorithms with explainable outputs. Replacing them with a model would make the system less accurate and far less defensible.

The place a model genuinely earns its keep is on the records that survive all six passes unmatched. That is a reading problem — what does this narrative mean, what family of failure does this resemble, what would an experienced analyst check first — and it is one that language models are extremely good at.

So the division of labour is: deterministic code decides what matches; the model explains what did not. Never the reverse. The moment a model is permitted to assert a match, you have a system whose output cannot be reproduced, and reconciliation whose output cannot be reproduced is not reconciliation.