I Don't Trust My Own AI Extractor: So I Made It Prove Every Number

Learn how to automatically verify your AI-extracted bank statement data using the built-in running balance consistency. No more hidden errors.

miércoles, 29 de julio de 2026 • 4 min read • Q2BSTUDIO Team

Verificación de extracciones de IA mediante balances contables

Artificial intelligence has revolutionized financial data extraction, but when it comes to numbers that drive business decisions, blind trust in a probabilistic model can be dangerous. At Q2BSTUDIO, where we develop custom software for sectors handling sensitive data, we have seen how a silent error in a PDF extraction can cost hours of reconciliation and, worse, lead to wrong decisions. That is why, when we build systems that process bank statements, invoices or accounting reports, we apply a fundamental principle: AI must prove every number it delivers.

The classic problem with visual language models is that they are incredibly accurate... until they are not. A 700-row document can have 699 perfect rows and one with a misread figure: 1,240.50 instead of 1,340.50. That row does not look wrong; it sorts, imports and sums to a plausible total. But a bookkeeper reconciling accounts weeks later finds a discrepancy of 100 units and no one knows why. That kind of error —plausible yet false— is far more dangerous than an obvious failure, because the latter gets caught while the convincing lie slips through.

The solution is not to train the model better, but to exploit the redundancy that the document itself offers. A bank statement prints a running balance next to each transaction. That balance sequence is a free verification signal: one row's balance plus the previous row's amount must equal the previous row's balance (or the opposite, depending on the bank's order). If all pairs satisfy that identity, the extraction is internally consistent. If a single figure is misread, the chain breaks. The interesting part is that this check is deterministic and external to the model: the AI cannot hallucinate a value that keeps the whole chain coherent because the constraint is overdetermined.

We implemented this mechanism in our custom software projects as a mandatory verification step. It is not enough for the model to return data; we need the tool to measure its own accuracy document by document. Thus we built a system that calculates the fraction of row pairs satisfying the identity, with a one-cent tolerance to absorb floating-point noise. If the score is below 0.99, the result is marked as UNVERIFIED and that label follows it in the downloaded file, the HTTP header and the JSON response. The user always knows whether they have a checked result or an uncertain one.

But there are important nuances. Banks do not order transactions the same way: some use descending order (newest first) and others ascending. Our solution evaluates both directions and keeps the one with the highest score, because a correct extraction reconciles in only one direction, while an erroneous one fails both. There are also formats that only print the balance at the end of each day, leaving most rows without that data. For those cases, we use a second verifier that accumulates amounts between two printed balances and checks that the final balance is correct, localizing the error in the exact segment where it occurs.

At Q2BSTUDIO we apply this philosophy beyond bank statements. In cloud AWS/Azure projects, for example, we integrate AI agents that process financial data but with deterministic verification layers that guarantee integrity before feeding BI/Power BI dashboards or cybersecurity systems. An AI agent can extract invoices, but a line-item-plus-tax-equals-total check ensures the sum adds up. The redundancy is in the document itself; we just need to leverage it.

The key is understanding that AI should not be the final word. A system that only says 'this is what I read' without being able to prove it is a black box. In contrast, a system that can show each number is internally consistent —that yesterday's balance plus today's movements equals today's balance— offers a transparency that clients value. It is not about replacing the accountant, but about giving them tools to trust the data without having to review every row manually.

The lesson applies to any domain where the cost of a silent error is high. If you are building a system that handles numbers someone will use to make decisions, look for the redundant signal already in your input. A bank statement has running balances; an invoice has a total that must equal the sum of line items; a ledger has double entries. That redundancy is a free, deterministic check you can wrap around your AI — and it lets you promise something few artificial intelligence tools can: if I am not sure, I will tell you, instead of guessing.

A BREAK?

Play for a moment before you go

OUR SERVICES

How we can help you

Do you have a project in mind?

Tell us your vision and we'll turn it into a software solution. Whatever the scope, we make your idea real.