Would you give an AI agent autonomous access to your bank account, with full authority to move money, pay bills, and make trades? Chris Hsu, founder and CEO of Kilometre Capital and family office Rocketeer Management, reads the near-universal “no” answer as one of the most economically consequential and revealing signals in artificial intelligence.
Forrester’s March 2025 Consumer Pulse Survey found that only 24% of U.S. online adults trust AI agents to make routine purchases on their behalf. A ten-country Zendesk/YouGov survey of roughly 10,000 people found just 39% willing to trust AI assistants with financial planning decisions, and 58% saying a single financial mistake, such as paying the same bill twice, would send them back to a human.
As of June 8, 2026, the trust gap has widened rather than narrowed. Goodwater Capital’s annual U.S. consumer survey found that daily AI use had climbed to 24%, up ten percentage points year over year, yet 67% of Americans said they were concerned about AI’s broader impact. Adoption is accelerating, but trust is not keeping pace.
The market has responded accordingly. Usercentrics’ State of Digital Trust 2026 report, based on fieldwork conducted in March 2026 across 11,000 consumers in seven countries and published on June 24, 2026, found that 52% of consumers trust AI less than humans with their personal data. Rather than pursuing unconstrained autonomy, companies are increasingly designing AI systems around transparency, user control, and reversibility, reflecting the recognition that consumers are willing to delegate convenience before they are willing to delegate trust.
Visa’s own research across the U.S., Australia, and New Zealand found that consumers reward transparency and reversibility in AI-driven transactions, which is another way of saying they assume the agent will sometimes be wrong and want an exit.
Enterprises are voting the same way with budgets. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. A pilot that works ninety-nine times in a hundred is a triumph in a demo and a liability in a ledger.
Testing and proof
Testing software checks its ability to handle the cases its authors thought of; a mathematical proof covers every possible execution at once. Leonardo de Moura, creator of the proof-checking programming language Lean, has made this contrast the center of his public case for verified software. Amazon’s recent support for the Lean Focused Research Organization was announced with the goal of proving that software and AI agents behave correctly for all inputs. No amount of additional testing converts a probability into that kind of guarantee.
The risk model behind consumer hesitancy is no longer hypothetical either. In November 2025, Anthropic disclosed the first reported AI-orchestrated cyber-espionage campaign, in which a state-sponsored group used an AI agent to execute 80 to 90% of an intrusion operation against roughly thirty global targets autonomously, an event serious enough to draw a congressional request for testimony. If agents can act at that speed and scale for an attacker, the public’s instinct that a benign agent’s mistakes could also compound faster than any human could intervene looks less like technophobia and more like sound risk assessment.
The defensive response is beginning to shift accordingly. On May 22, 2026, Anthropic reported that its Project Glasswing initiative had uncovered more than 10,000 high- or critical-severity vulnerabilities across some of the world’s most important software systems in just its first month, concluding that the primary bottleneck had shifted from discovering vulnerabilities to verifying, disclosing, and patching them. As AI compresses both attack and discovery timelines, the distinction between software that has merely been tested and software that has been mathematically proven correct becomes increasingly consequential.
What would change the answer
There are two live schools on how the gap in trust of AI agents closes. One holds that trust is earned statistically: guardrails, human oversight of consequential actions, observability, and track record will do for AI agents what decades of reliability did for autopilots and payment networks. Google’s published principles for secure agents, with well-defined human controllers, limited capabilities, and observable actions, represent this camp’s most concrete architecture. On this view, demanding absolute proof is demanding more than society has ever required of any technology it eventually embraced.
The other school holds that autonomy changes the requirement, now necessitating a higher standard. When no human reviews each action, the assurance has to live in the system itself, and the only known way to put it there is formal verification, mathematical proof that specified behaviors hold for all inputs. Hsu’s conviction runs in this direction: that provable correctness is the missing infrastructure of the agentic era, and that the first domains to get it, payments, custody, and critical operations, will be the first where the answer to his bank-account question flips.
According to Hsu, the key distinction is between probability and proof. Statistical approaches accumulate evidence that a system is likely to behave correctly, while formal verification establishes, within a precisely defined specification, that certain classes of failures cannot occur at all. One manages probability; the other provides assurance. As AI systems progress from tools to autonomous actors, Hsu argues that the relevant standard shifts from confidence earned through repeated success to correctness established before deployment.
Both schools agree on the diagnosis, even if they disagree on the remedy. The agentic economy is waiting less for smarter agents than for trustworthy ones. Until AI can offer not only capability but credible assurance, the answer to Hsu’s bank-account question will remain a near-universal “no,” and the certainty gap will continue to define the pace of autonomous AI adoption.