Prioritizing Architecture in Enterprise AI

Enterprises seeking effective AI solutions should focus on architecture rather than raw model power, according to Ricky Thakrar, head of sales and account management at Zoho. Speaking at the CIO 100 Leadership Live New York event last week, Thakrar emphasized that “smaller, smarter, safer” approaches consistently outperform expensive models deployed without proper context.

The Context Advantage

Thakrar explained that large language models (LLMs) often compensate for missing context by requiring more complex reasoning. When a model lacks sufficient information about a user’s history or account details, it must work harder to infer meaning - a less efficient approach than providing the necessary context upfront.

Zoho’s experience building AI agents demonstrates this principle: initial versions of their churn management agent provided inaccurate recommendations because they lacked crucial contextual data. While subsequent iterations improved accuracy, trust remained elusive as users continued to verify outputs independently.

Three Pillars of Effective AI Architecture

Thakrar outlined three key pillars for organizations seeking to build trustworthy AI:

1. Routing: Directing workloads to the appropriate model based on task requirements and context
2. Harness: Creating systems that integrate seamlessly with human workflows
3. Specialization: Employing focused models rather than relying on general-purpose giants

By prioritizing these architectural elements, companies can unlock greater value from AI while mitigating risks associated with inaccurate or untrustworthy outputs.