AI Pilots Are Proliferating, but Impact Is Stalling
Enterprise enthusiasm for AI has not yet translated into broad financial returns. IBM’s latest CEO study reports that only about a quarter of AI initiatives deliver the expected ROI, and just 16% have scaled across the enterprise. Other research echoes that imbalance: Morgan Stanley finds that only a minority of large listed companies report clear AI benefits, even as those that do enjoy materially higher cash-flow margins. At the same time, enterprise LLM budgets keep climbing, with one survey showing average spend per large company rising from around USD 4.5 million (approx. RM20.7 million) to USD 7 million (approx. RM32.2 million) over two years, and set to increase further. The pattern is familiar: lots of pilots, fragmented tooling and sparse governance. IBM is positioning its new enterprise AI orchestration portfolio as an answer to this scaling problem rather than yet another model or assistant.
An Operating Layer for Enterprise AI Orchestration
At its Think conference, IBM outlined what it calls an operating layer for AI, spanning agents, data, automation and hybrid cloud. The centerpiece is a next-generation watsonx Orchestrate in private preview, described as an agentic control plane that can deploy and manage AI agents from multiple sources under consistent policy and accountability. The aim is to shift from isolated AI assistants toward an integrated agent ecosystem that sits on shared AI agent infrastructure. IBM also highlighted IBM Bob, now generally available, as an agent-focused development partner with built-in security and cost controls for enterprise teams. Together, these elements are meant to provide a unified environment where AI workloads can be designed, tested and governed, rather than scattered across disconnected pilots. For organizations wrestling with scaling AI pilots, IBM is effectively offering a central nervous system to coordinate agent behavior, policies and lifecycle management.
Real-Time Data Integration as the Missing Scaling Ingredient
Many AI projects falter not because of model quality, but because they cannot reliably access trusted, up-to-date data. IBM is tying its operating layer directly to real-time data integration, leveraging its completed acquisition of Confluent to connect event streaming with watsonx.data, Kafka and Flink-based data flows. A new context layer adds semantic meaning and enforces governance at runtime, aiming to make AI decisions both explainable and auditable. IBM points to a proof of concept with Nestlé that delivered large cost savings and major price-performance gains on a global data mart spanning 186 countries, underscoring how tightly integrated data and compute can reshape economics. By embedding governance and lineage into streaming pipelines, IBM wants to ensure that AI agents operate on consistent, compliant information, reducing the operational friction that often derails scaling AI pilots beyond narrow proofs of concept.
Embedding Sovereignty and Compliance into the AI Stack
As organizations contemplate tens of thousands of agents running across hybrid environments, sovereignty and compliance become design constraints, not afterthoughts. IBM’s Sovereign Core targets this challenge by embedding policy enforcement at the infrastructure runtime level, with workload portability across on-premises and partner environments. Capabilities such as customer-operated control, in-boundary identity, encryption and data services, continuous compliance monitoring and automated audit evidence generation are designed to make governed AI execution the default. This matters because analyst forecasts are stark: Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027 due to rising costs, unclear value or weak risk controls, even as a typical large enterprise could run over 150,000 agents within a few years. IBM’s bet is that sovereignty-aware orchestration can prevent AI agent sprawl from turning into an unmanageable governance and regulatory liability.
From Discrete Assistants to Integrated Agent Ecosystems
The overall direction of IBM’s portfolio reflects a shift in how enterprises are thinking about AI. Early experiments focused on single assistants embedded in one workflow; the emerging reality is a mesh of specialized agents that must coordinate across business processes, data sources and infrastructure. IBM Concert extends this operating model into infrastructure and security operations, correlating signals across applications, networks and existing tools without demanding a complete tooling overhaul. Concert Secure Coder brings similar thinking to software development, embedding security management into developer workflows and generating targeted remediations. Yet surveys show a persistent value gap: while the vast majority of executives report some AI benefits, fewer than a third see significant ROI from generative AI or agents. IBM’s operating layer is framed as the connective tissue required to turn that fragmented promise into durable, enterprise-wide performance gains.