Who Audits the Auditors?
By Arunjay Katakam
On 9 March 2023, more than $40 billion left Silicon Valley Bank. Its supervisors had identified serious weaknesses, yet the Federal Reserve’s subsequent review found that they had been too slow to act on them. The failure was not simply a shortage of information. It was also a failure to turn warnings into decisions while there was still time. [1]
Banks and regulators have begun experimenting with AI. A machine can read more transactions, connect more signals and raise an alert sooner. It cannot, by itself, answer the harder question that SVB left behind: who is responsible for acting?
The usual objection to AI in risk and compliance is that it is a black box. A model produces a finding, but the people affected may be unable to see how it got there. That is a serious objection. It should also make us examine the system we have now. Human supervision can be opaque too: judgments sit in case files, warnings move slowly through committees, and the reasons for waiting are not always clear to outsiders.
The question is therefore bigger than whether an AI model is accurate. Who checks the checker, and what happens when it is wrong?
What the experiments show
Regulators are already helping firms test the technology. The FCA opened applications for its Supercharged Sandbox in 2025.
In September its Handbook went live as a machine-readable rulebook that an AI system can navigate, and by July 2026 a second cohort, includes 21 organisations, with Anthropic supporting the programme. These are controlled tests, not evidence that regulators have handed supervision over to machines. [2],[3]
The Bank for International Settlements has produced a more concrete result in financial crime detection. In a proof of concept, Project Aurora found up to three times more complex money-laundering schemes and reduced false positives by up to 80 percent compared with the approaches it tested. Project Hertha then examined how payment-system data might help identify suspicious patterns while using a limited set of data points. Promising results in a test environment still have to survive contact with live institutions, messy records and people whose payments are at stake. [4]
I find the direction compelling. A transaction can cross several institutions before any one of them sees enough to understand its pattern. Software may be able to connect those fragments much faster than a team working through separate alerts. Programmable payments could also make some checks possible before funds move, rather than months afterwards.
But seeing more is not the same as understanding more. A system that examines every transaction can still misunderstand an ordinary customer. At scale, it can make the same mistake thousands of times.
The scene I wrote to test it
In The Future of Finance is Open, I put six national banks on shared, programmable ledgers and give them automated compliance tools. Their compliance officers are hostile to the pilot. They suspect the machines are being brought in to replace them.
Within seventy-two hours, the system finds a pattern of transactions moving through the banks’ subsidiaries. The activity had passed through years of conventional reviews without anyone seeing the whole network. Every bank finds something similar. It is the sort of result that can change minds.
But the character I trust most in that room is the sceptic: a former head of a financial intelligence unit. He asks the question the demonstration cannot answer. What happens when the system is wrong?
In the book, his agency finds three recurring failures. One tool mistakes transfers between a customer’s own accounts for sophisticated layering. Another fills gaps in noisy data with a convincing account of a crime that never happened. A third reaches a correct finding but cannot show an investigator how it got there.
I wrote those failures into the story because success in detecting crime is only half the test. A system must also let someone challenge its findings, correct its mistakes and account for the harm it causes. In a regulated industry, an unexplained decision can be a liability even when it happens to be right.
Automation needs an owner
I would set five conditions before allowing an automated system to make or trigger consequential supervisory decisions.
First, people independent of the vendor and the institution using the model must test it against realistic cases and changing conditions.
Second, every decision needs a traceable record of the data, rules and model version involved.
Third, a staffed human team must be able to pause an action and review an appeal in time to make a difference.
Fourth, model changes need testing, approval and a workable way to roll back.
Fifth, the institution must say plainly who corrects an error and who bears responsibility for the harm it causes.
A human override sounds reassuring on paper. It is useful only if someone has the information, authority and time to use it. Otherwise, the person becomes a rubber stamp on a decision made out of their control.
That matters most to people who cannot easily wait for a correction: a migrant whose remittance is frozen, a small trader who needs payment to release goods, or a borrower whose thin credit file gives the system little context. A mistaken decision repeated consistently may do more damage than a human error precisely because it travels so quickly.
And quantum?
I am often asked whether quantum computing is already improving financial risk models. I have not seen a compelling case that it is changing day-to-day compliance decisions yet.
The more immediate work is preparing the cryptography on which digital finance depends for a future in which sufficiently capable quantum computers could break widely used signatures. If we expect to rely on verifiable credentials, tokenised assets and automated checks, we need to know that their underlying proofs will remain trustworthy. That migration deserves attention now, without pretending that quantum computing has already solved financial crime.
The questions to ask
Before an AI system enters a live compliance or supervisory process, I would ask three questions in plain language: Who independently checks its work? Who can stop it quickly? Who puts things right when it harms someone?
If the answers depend on a vendor presentation or a committee that meets next month, the system is not ready for the responsibility it has been given.
I believe automation can make oversight faster and more perceptive. SVB showed how costly a delay can be. The lesson I take from it is that finding a warning sooner matters only if somebody is accountable for what happens next.
Sources:
[1] Board of Governors of the Federal Reserve System
[2] FCA Supercharged Sandbox
[3] FinregE and FCA
[4] The BIS Innovation Hub - Project Aurora
