Apple: MLX, Core ML and Apple Silicon

Models and offerings #

Who they are

This hub is for engineers whose only GPU is in a MacBook, and for anyone shipping a model inside an iOS or macOS app. Apple is a hardware company that built machine learning into its chips: the Neural Engine arrived with the A11 in 2017, Core ML the same year, and the M1 in 2020 put a capable GPU and a large pool of unified memory in a laptop. MLX, an open-source array framework from Apple’s machine learning research group, appeared in December 2023 and became the practical way to run and fine-tune open language models on a Mac.

Apple’s position in the field is unusual. It publishes no frontier model, sells no model API, and its interest is in models that run locally on devices it sells. That makes it the strongest platform for private, offline inference and the weakest for large-scale training. The efficient ML topic covers the trade-off.

Tooling and APIs

For experiments and local inference, pip install mlx mlx-lm. MLX 0.32.4 (14 October 2026) is MIT licensed, has Python, Swift and C++ bindings, and now has a CUDA backend. Distributed training across several Macs uses the JACCL backend over Thunderbolt with RDMA, which Apple presented at WWDC 2026 alongside Metal 4 support. The mlx-community organisation on Hugging Face publishes 4-bit and 8-bit conversions of most open models.

# snippet
from mlx_lm import generate, load

model, tokenizer = load("mlx-community/SmolLM2-135M-Instruct-4bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "In one sentence, what does a learning rate control?"}],
    add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=40))

MLX is not in the Field Guide pipeline environment, so this is a syntax-checked snippet. The PyTorch MPS backend needs macOS 15 or newer, gained FP8 in 2.15, and runs the LoRA fine-tune project.

For shipping in an app there are now three Apple routes. Core ML, with coremltools (BSD-3) to convert a PyTorch model, is the mature path and runs on CPU, GPU and Neural Engine. At WWDC in June 2026 Apple announced Core AI, which it describes as built into the OS and purpose-built for Apple Silicon, with ahead-of-time compilation, fine-grained memory control and Python tools for converting PyTorch models. Apple’s session material positions Core ML as remaining for tree ensembles, regression and other classical models while new generative capability lands in Core AI. The Foundation Models framework, introduced in 2025, is a Swift API to the on-device model behind Apple Intelligence (Apple reported about 3 B parameters for the 2025 on-device model, and the current model’s size is not stated on the developer page), to the larger model on Private Cloud Compute, and since 2026 to any provider that adopts its Language Model protocol, including Claude and Gemini cloud models. Apps with under 2 million first-time downloads get the Private Cloud Compute model with no cloud fee, according to Apple.

All of it is free with the OS, and shipping on the App Store needs the paid developer programme. Pain points: Core ML conversion fails on unsupported operators with thin error messages, the Neural Engine is reached only through Core ML and Core AI (MLX and PyTorch use the GPU), and Core AI is new enough that it needs the OS versions released this autumn.

Migration notes

From CUDA: there is no CUDA on a Mac. PyTorch code runs on MPS with device="mps", with operator gaps that fall back to CPU, and anything using torch.cuda APIs, Triton kernels or FlashAttention needs changing. MLX is the better fit for LLM inference and fine-tuning, and its API is close enough to NumPy and PyTorch that porting a training loop takes an afternoon. Going the other way, MLX’s CUDA backend runs MLX code on NVIDIA hardware.

Model formats: Hub safetensors convert to MLX with mlx_lm.convert (which also quantises), to GGUF for llama.cpp with the llama.cpp scripts (llama.cpp’s Metal backend is the other common way to run models on a Mac, and Ollama uses MLX on Apple Silicon), and to Core ML with coremltools. Core AI conversion tooling is described by Apple as Python tools for PyTorch models, so check the current documentation for its model format. Each conversion is one-way.

Naming: Apple Intelligence is the consumer feature set, Foundation Models the framework, Apple Foundation Models the models, and Private Cloud Compute the server side.

Watch for

Lock-in is to the hardware rather than to a service. MLX runs on Apple Silicon and, newly, CUDA. Core ML and Core AI models run only on Apple devices. Nothing here helps you train at scale: a Mac Studio with 512 GB of unified memory can hold a large model, but its GPU is far slower than a data-centre part and multi-node means a Thunderbolt cluster of a few machines.

Framework churn is the current risk. Core AI was announced in 2026 as the successor to Core ML for generative models, its documentation is young, and third-party coverage disagrees about whether Core ML is deprecated. Apple’s own wording is that Core ML remains for traditional models. An existing Core ML pipeline keeps working, but new work should check which framework Apple is investing in.

The Foundation Models framework runs only on devices and OS versions that support Apple Intelligence, which excludes older hardware and some regions, and the on-device model is small, so test your task on it rather than assuming parity with a hosted model. Private Cloud Compute is still a cloud dependency, under Apple’s privacy terms. MLX and coremltools send no telemetry. Apple publishes no reproducible Neural Engine benchmarks, so measure your own model with Instruments.