Money Doesn’t Get to Be “Probably Right”
By Rafał Kucharski, Principal Fintech Architect at The Software House
A few months ago, my team at The Software House gave a chat interface the ability to move money. The first message someone typed into the test build was: “Send €500 to John.” Four words. No account number, no currency, no confirmation of which John. The model, being a model, answered with total confidence. It just wasn't the right answer.
That gap between confident and correct is the real fault line in agentic payments right now. Underneath the regulatory or trust issues sits an engineering one that gets less attention. How do you build when the model deciding who gets paid, how much, and in which currency answers with the same confidence whether it's right or wrong?
The industry isn't waiting to find out. In March 2026, Santander and Mastercard ran Europe's first live end-to-end payment executed by an AI agent inside a regulated banking framework. It was a controlled pilot, but moved real money on live infrastructure, not a stage demo.
That was the same question my team had to answer, on a smaller scale. We spent less than a year building an agentic payment system for Bakkt, a digital-asset infrastructure company. We turned its product, Zaira, into a single conversational agent that covers onboarding, KYC, funding, and transfers across more than 60 countries.
The decision before the build
Before any of that shipped, we made the hardest call of the project – deciding upfront, on everything the agent would never be allowed to do on its own.
Three commitments came out of that conversation. Everything built afterward had to fit inside them, not the other way around.
Data the model never touches
Identity documents were the clearest case. A KYC flow deals in passports, selfies, and proof of address, the kind of data you do not want near an LLM's context window.
The fix wasn't a clever prompt. It was a pipeline that intercepted anything sensitive and swapped it for a unique encrypted identifier before the agent saw the message. Once the model finished responding, the pipeline matched each identifier back to its real value and restored it.
Money can't move by itself
No matter how well the agent understood the phrase "send €500 to John for the trip," the system never allowed that sentence to be processed as a transaction on its own.
Every parameter the user confirmed on screen, the recipient, the amount, and the currency, gets hashed on a path separate from whatever the model produced.
Call it the anti-hallucination guardrails: two full payloads, generated on separate paths, either match field for field or they don't.
If they don't, the transfer is blocked before it reaches a bank or a card network.
Guardrails don't double-check the model's judgment, but skip trusting that judgment altogether. The guardrails don’t ask whether an answer looks plausible (which is still a call a model can get wrong) but whether two payloads, produced on separate paths, match parameter by parameter. That isn't a judgment call.
Where the agent's authority ends
The guardrail closed one gap. The next was scope: a single model handling onboarding, KYC, funding, transfers, and support gets to decide, on its own, what counts as its job.
Instead of one model deciding everything, Zaira runs on specialized agents, each with its own narrow mandate and its own narrow set of tools it's allowed to call. A router reads the request and hands it to the agent built for that job. A funding agent cannot execute a transfer. A support agent cannot touch KYC data. The boundaries were set before the system shipped, not negotiated by the model in the moment.
For the user, this whole process is still one simple conversation.
Safety lives in the guardrails
None of this is new. Isolating sensitive data, splitting work across narrow agents, and checking results without trusting the model that produced them, these ideas already exist across the industry. That makes sense, given how much money now moves this way.
The lesson worth taking: safety doesn't live inside the model, but in the boundary you build around it. Without that boundary, treat the agent as a confident storyteller, not a trusted decision-maker.
Before any team lets a model near a payment flow, ask what guardrails stand between AI’s answer and someone's money. Would that guardrail hold if the model was wrong?
The same logic applies anywhere an agent can make a lasting change: look at the framework and ask whether it leaves room for something reckless or unexpected.
Sounding right is free now; every major model can do that. What still has to be earned is a guardrail strong enough to survive being wrong. In banking, "probably right" was never good enough.
