The AI Control Gap: Companies Adopting AI Must Learn What Wall Street Learned The Hard Way
Reported by Forbes1 source ↓
ApyHub CEO Samuel Kaluvuri argues that companies using autonomous AI agents need controls over what agents can access and do, and must be able to monitor and stop their actions.
Key points
- Kaluvuri says controls should limit agents’ actions, stop them in progress and record what they did
- He cites Knight Capital’s 2012 software failure, which lost more than $460 million in 45 minutes
- Buyers may control agent policies and logging themselves, or share control with vendors
- The article says the Financial Stability Board warned in June 2026 that agent autonomy can amplify fast-moving risks
Details
Kaluvuri argues that traditional approvals are not enough for software that can take many actions without human review. An approval for a broad task, he writes, may not specify which systems an agent can access, what data it can move or how it should respond to unexpected cases.
He points to financial trading as a precedent: Knight Capital lost more than $460 million in 45 minutes after faulty code went live in 2012. Kaluvuri says the response to risks from fast automated trading included limits enforced before trades, kill switches and real-time monitoring. He also cites the SEC market access rule, which requires covered firms to keep direct and exclusive control over relevant risk controls, even when they use third-party technology.
For companies buying agent platforms, Kaluvuri says the key questions include where runtime controls are enforced and who can inspect and change them. He argues that buyers should extend their own access policies, transaction limits and logging to agents rather than assume vendor defaults or assurances are sufficient.
The article says the Financial Stability Board warned in June 2026 that agent autonomy could amplify risks that materialize quickly. Kaluvuri says that when suitable controls are not in place, keeping a workflow under human supervision may be preferable to handing it fully to an agent.
Written from coverage as of
Images
- Forbes