Building the foundations for agentic AI at scale

A house is only as strong as its foundation. That’s what companies are quickly coming to understand about agentic AI as well. Nearly two-thirds of enterprises worldwide have experimented with agents, but fewer than 10 percent have scaled them to deliver tangible value.  Shaky data is often to blame; eight in ten companies cite data limitations as a roadblock to scaling agentic AI. Addressing this issue is a core element of building a solid capability foundation—and that’s what distinguishes companies that create value from AI from those that don’t.  While companies have often muscled through issues of fragmented and siloed data, those issues are impossible to manage at scale. Inconsistent governance has just increased the challenge of preserving data context while enforcing access control, lineage, and auditability.

Success with agentic AI depends on a data architecture that can support increasing levels of autonomy, coordination, and real-time decision-making. This often looks like modular, interoperable frameworks that give agents reliable access to the data they need to operate safely (see sidebar, “Seven data architecture principles that enable scale”). While gen AI has already shown the need for data access control, lineage, and traceability, agentic platforms place greater operational pressure on these foundations. Because agentic AI coordinates multiple models and data sources continuously, often without human intervention, it requires tighter, more automated governance to ensure reliability and control at scale.

Two agentic archetypes are emerging: single-agent workflows, where one agent uses multiple tools and data sources sequentially; and multi-agent workflows, where specialized agents collaborate through shared knowledge graphs and fine-grained data access. Both require consistent, interoperable data, without which agents could break down.

How to prepare data for agentic AI.  To enable a scaled transformation into an agentic organization, companies can start by building foundational data capabilities. This requires not just a technology reboot, but also an organizational one. That’s because a company’s data strategy and operating model is just as important as its underlying data quality and architecture. Success depends on taking four coordinated steps that link strategy, technology, and people:  

Step 1: Identify high-impact workflows to “agentify.” Organizations can identify a small number of high-value, end-to-end workflows where increased autonomy could unlock impact. Building on established approaches to building data products, leaders can prioritize agentic use cases based on value potential, feasibility, and strategic fit before scaling more broadly.

Step 2: Modernize each layer of the data architecture for agents. Rather than rebuilding everything from scratch, leaders can modernize existing platforms to support interoperability and governance across systems. While some may be tempted to lean on advancements in AI to shortcut data architecture best practices, the strongest organizations build modular, evolutionary architectures with components that can be replaced as new technologies emerge.

Step 3: Ensure that data quality is in place. Organizations must move from periodic data cleanup to continuous, real-time quality management. They can do this while ensuring that both structured and unstructured data, as well as agent-generated outputs, meet consistent standards for accuracy, lineage, and governance.

Step 4: Build an operating and governance model for agentic AI. Scaling agentic AI requires rethinking how work gets done. Human roles are shifting from execution to supervision and orchestration of agent-driven workflows. In a hybrid human–agent work environment, clear governance is essential to allow agents to operate transparently and safely at scale.

show less