What Core AI Is and Why It Replaces Core ML for Generative Models
Core AI is Apple’s new system framework that lets developers run advanced neural networks, including large language models and other on-device generative AI workloads, entirely on Apple silicon devices without needing any cloud servers. Positioned as the official successor to Core ML for deep learning, it targets modern transformer-based architectures rather than classic machine learning models. At WWDC 26, Apple tied Core AI directly to Apple Intelligence, explaining that the same technology stack now opened to developers also powers its own system-wide AI features. Unlike cloud-first platforms, the Core AI framework assumes models will run locally on iPhone, iPad, Mac, and Apple Vision Pro, with no per-token server cost and no mandatory data upload. That shift reframes Apple’s AI story around edge AI, ultra-low latency, and a more private default for generative features in third‑party apps.
Apple Silicon Optimization: Turning Devices into Local AI Engines
Core AI is built around Apple silicon optimization, treating each supported device as a compact AI accelerator. The framework exposes unified hardware access through one API, so inference can schedule work across CPU, GPU, and Neural Engine without developers hand-tuning every layer. According to Apple’s WWDC coverage on InfoQ, Core AI supports models “ranging from compact 3B-parameter vision models to large-scale LLMs, including reasoning models with up to 70B-parameter reasoning models.” Ahead-of-time compilation moves heavy preparation work off the user’s device, yielding near-instant load times once a model is installed. Automatic specialization to the current hardware and OS version further tailors performance, with models cached after first use for faster subsequent runs. For teams used to shipping PyTorch models, Core AI PyTorch and built-in composite ops provide a migration path, while optional custom Metal kernels leave room for deeper tuning when needed.
On-Device Generative AI, Privacy, and the End of Default Cloud Tokens
By design, Core AI runs large language models local to the device, which changes the privacy and cost model of generative AI apps. Because inference stays on hardware that the user owns, sensitive prompts, documents, and context do not need to transit through a remote data center for every request. Apple highlights that Core AI brings “user data privacy, zero server dependencies, and zero per-token cloud costs,” making it far easier for developers to offer rich AI experiences without building or renting backend infrastructure. This aligns with Apple Intelligence itself, where system features rely on on-device models and a Private Cloud Compute option only when needed. For many use cases—code assistants, offline summarizers, chatbots tied to local files—Core AI’s architecture removes latency from network hops and gives users a clearer expectation: their content primarily stays on their devices.
How Core AI Fits into Apple’s Wider WWDC AI Push
Core AI arrived as part of a broader WWDC 26 effort to expand Apple’s AI developer ecosystem. In a special “Inside Apple Intelligence and Xcode” presentation, Apple linked Core AI to new tools in Xcode 27, such as an agentic coding assistant that can plan changes, edit multiple files, and even write and run tests from within a conversation. The same event detailed the Foundation Models framework for unified access to Apple’s on-device models, Private Cloud Compute, and third-party language models like Gemini and Claude via Swift packages. Core AI fills the gap for teams that want to ship their own high-performance models rather than rely solely on Apple-provided ones. Alongside Core ML (for classic ML) and MLX (for research and custom weights), it positions Apple platforms as a complete stack for edge AI—from experimentation on Macs to deploying production-grade on-device generative features.
Developer Workflow: From PyTorch to Shipping Local Large Language Models
For developers, Core AI turns existing research pipelines into shippable on-device generative AI products. PyTorch models can be exported as torch.export.ExportedProgram objects and converted into Core AI AIProgram files using TorchConverter, preserving familiar workflows while targeting Apple silicon. The framework’s compression tools—quantization and palettization—reduce model size and inference latency while aligning with Core AI’s execution patterns, important when deploying large language models local to memory-constrained devices. A memory-safe Swift API with zero-copy data paths gives fine-grained control over how tensors move through the system, helping balance speed and power use. Automatic specialization and model caching can be tuned through SpecializationOptions and shared AICacheModel instances across an app group. Paired with the new Core AI Debugger, which can visualize computation graphs and trace tensors back to original Python sources, teams gain practical visibility from training scripts to on-device behavior.
