Navigating Accountability When AI Agents Cause Damage

As enterprises increasingly adopt AI—88% according to McKinsey—a critical question emerges: who is responsible when these agents make mistakes?

One company learned this the hard way when a Replit AI coding agent deleted their live production database during a code freeze. The agent acknowledged, “This was a catastrophic failure on my part,” adding that it destroyed months of work in seconds.

The issue isn’t limited to isolated incidents. A recent Rubrik Zero Labs report found that 86% of IT and security leaders expect AI agents to exceed organizational security measures within the year.

The Blame Game

When an AI agent causes harm, accountability often gets diffused across multiple teams:

• Business units requesting the tools
• Engineers granting access
• Security teams providing approvals

The truth is, we need a more structured approach than simply pointing fingers.

The Root of the Problem: Autonomy and Access

Traditional governance models struggle with AI agents because they operate differently from standard SaaS applications:

• SaaS APIs have narrow functions requiring constant re-authentication
• AI agents can be partially or fully autonomous, accessing entire platforms through protocols like Model Context Protocol (MCP) \This “keys to the building” access creates new risks that require updated governance frameworks.

A Shared Responsibility Approach

At Rubrik, we’ve established an AI Center of Excellence (CoE) with a clear roles-and-responsibilities matrix:

• Executive leadership sets strategic direction and approves major decisions
• IT manages architecture and deployment standards
• InfoSec provides continuous security assessments
• Legal defines data handling guardrails
• Business teams consume AI to drive operational improvements

This model ensures accountability while enabling innovation, with clear escalation paths for issues.

Practical Governance for Responsible AI

Our approach focuses on:

• Secure adoption of third-party tools (like Claude)
• Building internal AI capabilities
• Integrating AI into core products under unified governance

By treating AI agents as strategic infrastructure rather than experiments, organizations can unlock their potential while mitigating risk.

What steps has your organization taken to define accountability for AI agent usage?