What Core AI Is and Why It Replaces Core ML for Generative Models
The Core AI framework is Apple’s new on-device AI stack that lets developers run large language models and other generative AI models locally on Apple Silicon, replacing Core ML for neural and transformer-based workloads while keeping data processing on user devices instead of in the cloud. Announced at WWDC 26 as the official successor to Core ML in this space, Core AI is the foundation of Apple’s own Apple Intelligence features and is now being opened up so developers can build what Apple calls “custom intelligence” into their apps. Unlike traditional client–server AI setups, Core AI is designed around zero server dependencies and zero per-token cloud costs, shifting both inference and most preparation work to the user’s hardware. This design directly targets concerns about latency, reliability, and privacy that come with cloud-dependent AI services.
On-Device Generative AI and Local Language Models Across Apple Platforms
Core AI is built to run on-device AI models that span from compact vision networks to large-scale local language models, all executing on Apple Silicon. According to InfoQ, the framework supports “compact 3B-parameter vision models” and “reasoning models with up to 70B-parameter reasoning models” across iPhone, iPad, Mac, and Apple Vision Pro. This range makes Core AI suitable for everything from offline image understanding to full-featured assistant-style applications. Because models are deployed locally, responses are not tied to server availability or network quality, helping developers deliver predictable performance in everyday conditions. Core AI’s role as the underpinning of Apple generative AI experiences means third-party apps can now share many of the same building blocks as Apple’s own features. That alignment may reduce fragmentation for users, who increasingly expect consistent AI behavior across devices and apps.
Apple Silicon Optimization: Unified Hardware Access and AOT Compilation
A central promise of the Core AI framework is deep Apple Silicon optimization. Core AI presents a unified hardware interface so developers can target one API while the runtime schedules work across CPU, GPU, and the Neural Engine. This unified access is paired with a memory-safe Swift API that supports zero-copy data paths and fine-grained control over inference memory, helping keep latency low even for larger on-device AI models. Ahead-of-time compilation further shifts heavy lifting away from user interaction: models are compiled before runtime, which Apple says yields near-instant load times once a model is cached. Core AI also specializes each AIModel to the current hardware and OS version when it is first loaded into the cache, so later runs benefit from device-specific tuning. Developers can manage this specialization lifecycle through options and cache APIs to pre-warm or share compiled models across app groups.
Developer Workflow: From PyTorch to Core AI and Model Compression
Apple positions Core AI as a practical bridge between existing research workflows and production-grade on-device AI. Developers can convert PyTorch models into Core AI via Core AI PyTorch by exporting a torch.export.ExportedProgram and transforming it into an AIProgram with TorchConverter().add_exported_program(ep).to_coreai(). For more advanced use cases, Core AI exposes composite operations such as attention, RoPE embeddings, RMSNorm, and gather-matmul, while still allowing registration of custom lowering functions or authoring custom Metal kernels for low-level optimization. A crucial deployment step is compressing models for Apple hardware, using techniques like quantization and palettization to reduce memory footprint, latency, and power use in line with Core AI’s execution patterns. Together with Core ML for classic non-neural ML and MLX Swift for custom weight experimentation, Core AI completes a three-layer stack that steers developers toward efficient on-device generative AI.
Strategic Shift: Privacy, Latency, and Apple’s Position Against Cloud AI Rivals
By centering the Core AI framework on local language models and on-device generative AI, Apple is making a clear strategic play against cloud-dependent AI competitors. Core AI’s promise of user data privacy, zero server dependencies, and zero per-token cloud costs directly addresses recurring concerns about data exposure, recurring fees, and network latency in AI features. This design fits Apple’s broader integration strategy, in which Apple generative AI features live across iOS, macOS, and Apple Vision Pro, and third-party apps can tie into the same on-device capabilities. While Core AI shares the stage with Core ML and MLX Swift, community feedback highlighted that Core AI “makes it easier to incorporate high-performance LLMs” but that its long-term impact will depend on how the official APIs and community evolve. For developers, the bet is clear: embrace Apple Silicon optimization, and ship AI that is fast, private, and offline-first.
