Managing AI Risk in Real-Time

The Bank of England’s recent warnings about autonomous AI agents have prompted financial firms to reassess their risk management strategies. While kill switches remain a crucial safety net, experts argue that continuous monitoring and governance are essential for responsible AI deployment.

Cameron Etezadi, CTO at LaunchDarkly, emphasizes that the nature of failure has changed with AI. Traditional software failures were largely predictable through testing, while AI systems exhibit dynamic behavior—generating different outputs from the same inputs or drifting over time even without code changes. This inherent variability is not necessarily a defect but rather a characteristic that enables AI’s unique capabilities.

The Need for Runtime Controls

Etezadi draws an analogy to driving: “Kill switches are like emergency brakes—essential in critical moments, but not how you manage most journeys.” Instead, he advocates for runtime controls that provide ongoing visibility and adjustment capabilities. These enable organizations to:

• Monitor outputs and steer behavior while systems are live
• Reduce permissions or transaction limits when needed
• Reroute workflows to safer models or human review
• Adjust resource allocation based on demand

For irreversible actions like payments, the focus should be on containment rather than stopping a system after harm has occurred. By intervening early, organizations can limit the “blast radius” of potential errors.

Beyond Rollout Percentages

While gradual rollouts help identify common issues, Etezadi cautions that they are not foolproof. A system might appear stable at low exposure levels simply because it hasn’t yet encountered the specific conditions that trigger failure. For instance, infrastructure might handle a small volume of connections but quickly become overwhelmed by a sudden surge.

Instead, teams need comprehensive monitoring tied to business outcomes—tracking metrics like load, latency, cost, quality, and error rates—to ensure systems remain healthy across all operating conditions.