Apple Shows How to Run AI Agents Locally on Mac With MLX [Video]

Apple is giving developers a closer look at how to run agentic artificial intelligence workflows entirely on-device using its MLX framework. A recent WWDC session outlined a local software stack that allows AI agents to operate directly on Mac hardware, eliminating the need for cloud processing or external API keys.



The setup relies on four software layers. At the foundation is MLX, Apple's open-source array framework built specifically for the unified memory and hardware acceleration of Apple Silicon. Above that sits MLX-LM for model loading and fine-tuning, followed by the MLX-LM Server. Because this server layer exposes local models through an OpenAI-compatible HTTP interface, Apple describes it as a drop-in replacement for cloud LLM services. This allows top-level agent frameworks like OpenCode or Xcode 27 to communicate with locally hosted models using the same protocol commonly used for cloud-based AI systems.


In demonstrations, Apple showed an agent fetching pull requests from GitHub and summarizing code changes locally. The company also demonstrated an agent building a functional SwiftUI application from scratch, compiling the project, and fixing errors along the way until the app ran successfully.

Keeping these autonomous workflows moving requires heavy compute, especially since agents repeatedly process prompts and tool outputs to figure out their next move. To handle the load, Apple highlighted that the M5 chip features dedicated neural accelerators capable of matrix multiplication four times faster than the M4. The MLX framework taps into those specific hardware gains to deliver a similar 4x speedup in prompt processing. The software also utilizes continuous batching, grouping requests from parallel sub-agents so they process simultaneously on the GPU rather than stalling in a queue.

For massive models that need more unified memory than a single machine can offer, MLX now supports distributed inference. Developers can spread a single heavy model, such as a 1.6-trillion parameter model requiring over 800GB of RAM, across multiple Macs connected via Ethernet or Thunderbolt. Using Thunderbolt RDMA for low-latency communication, Apple demonstrated up to a 3x speedup when pooling four machines.

This ability to efficiently run capable AI models locally is a big reason why Apple is seeing a surge in demand for the Mac mini and Mac Studio. As always, you can track Mac availability and pricing using the iClarified Mac Price Tracker.