AI Spend Is Surging, But Enterprise ROI Is Stuck in Pilot Mode
Enterprises are pouring money into generative AI, yet most initiatives remain trapped in proof-of-concept. IBM’s own CEO study finds only around a quarter of AI projects deliver expected returns, and just 16% have scaled across the enterprise. Other analysts point to the same mismatch between enthusiasm and payoff: many large companies now report at least one AI benefit, but those seeing clear, cash-flow-level gains are still the exception. At the same time, capital expenditure on AI infrastructure is accelerating, driven by investments in compute, data centers and power. Inside Global 2000 enterprises, average large language model spend has climbed sharply over the past two years and is expected to rise again. This widening gap between AI investment and measurable business value is the strategic opening IBM is targeting, as organizations search for enterprise AI infrastructure that can reliably move AI pilots into governed, production-grade systems.
IBM’s Operating Layer: Agents, Data, Automation and Hybrid Cloud
At Think 2026, IBM outlined an operating layer it argues is now required to scale AI: a unified stack spanning agents, data, automation and hybrid cloud. Rather than offering more isolated tools, IBM is positioning this as an operating model shift that rewires how enterprises run their businesses with AI embedded throughout. The idea is to standardize how organizations design, deploy and manage AI workloads, from early experimentation through large-scale production. By combining an agent orchestration platform with real-time data access, AI operations management and sovereignty controls, IBM aims to reduce the friction that usually appears between pilot and production. The company’s leadership stresses that the enterprises pulling ahead are not simply deploying more models; they are systematizing the way AI interacts with processes, platforms and people. IBM’s pitch is that an integrated operating layer can close this gap more effectively than ad hoc tooling and one-off pilots.
Agent Orchestration: A Control Plane for the Coming Agent Explosion
A central plank of IBM’s strategy is next-generation watsonx Orchestrate, described as an agentic control plane now in private preview. As enterprises experiment with task-specific AI agents from multiple vendors, IBM argues they need a consistent way to deploy, monitor and govern them. The platform is designed to let organizations onboard agents from different sources while enforcing common policies, security rules and accountability. IBM is also highlighting IBM Bob, now generally available, as an in-house development partner for teams building agents with built-in controls around security and cost. This focus on an agent orchestration platform comes as analysts warn about looming agent sprawl: large enterprises could be running massive numbers of agents within a few years, yet only a small minority feel prepared with adequate governance. Without a coherent control plane, rising costs, unclear business value and unmanaged risk could derail many agentic AI initiatives.
Real-Time Data and AI Operations Management Close the Infrastructure Gap
Beyond agents, IBM is attacking the data and operations bottlenecks that often block AI pilot scaling. Leveraging its completed Confluent acquisition, IBM is tying real-time event streaming into watsonx.data and Kafka- and Flink-based pipelines. A new context layer adds semantic meaning to this streaming data, applying governance dynamically at runtime and supporting more explainable AI decisions. Early evidence from a proof of concept with Nestlé indicates potential cost and performance advantages at global scale. On the AI operations management side, IBM Concert aims to extend the same operating model into infrastructure and security operations. Now in public preview, Concert is built to ingest and correlate signals across applications, infrastructure and networks, without forcing customers to rip and replace existing tools. Concert Secure Coder adds developer-focused automation, generating remediation suggestions for vulnerabilities across code, operating systems, middleware, packages and images.
Sovereignty, Governance and IBM’s Bid for the Enterprise AI Middle Layer
Governance and sovereignty are emerging as decisive factors in whether AI pilots can safely scale. IBM Sovereign Core addresses this by embedding policy directly at the infrastructure runtime level, while maintaining workload portability across hybrid and partner environments. Capabilities include customer-operated control, in-boundary identity, encryption and data services, continuous compliance monitoring, audit evidence generation and governed AI execution. With analysts warning that a significant share of agentic AI projects may be canceled due to weak risk controls and unclear value, IBM is betting that sovereignty-by-design will become a baseline requirement. The broader operating layer strategy positions IBM as a neutral middle layer above heterogeneous clouds and tools, targeting enterprises now ready to graduate from experimentation to deployment. If successful, this approach could turn IBM’s orchestration, data and governance stack into the connective tissue of enterprise AI infrastructure for the next wave of scaled deployments.