i've been running local ai on macs for two years. ollama, llama.cpp, python venvs that break every update, cuda drivers that don't exist on apple silicon — i've fought all of it. and then i found MLX.
MLX is apple's machine learning framework, built specifically for their chips. not ported. not adapted. built for the hardware you already have. 25K+ stars on github, actively maintained by apple's ML research team, and gaining adoption fast in the apple silicon developer community [1].
the shared memory thing
here's the part that actually matters: on intel/nvidia setups, the CPU and GPU have separate memory pools. every computation means copying data back and forth across the PCIe bus. that's latency. that's heat. that's wasted cycles. an NVIDIA RTX 4090 has 1TB/s bandwidth to its own VRAM — but the CPU→GPU transfer is limited to ~64 GB/s across PCIe 4.0 x16.
apple silicon has unified memory architecture. CPU and GPU share the same physical pool. the M4 Max delivers 546 GB/s bandwidth to a single shared memory — no PCIe bottleneck. MLX exploits this directly — no copies, no transfers, no data movement overhead. the model weights live in memory once, and both processors access them simultaneously [1].
the result? on an M4 with 128GB of unified memory, you can run a 70B parameter model locally. not a cloud proxy. not a quantized toy. a real model, on your machine, at speeds that make you forget it's not chatGPT.
mlx-lm — the one you actually use
mlx-lm (4.7K stars) is the serving layer. you point it at a model, it starts an openAI-compatible API server on your machine. same format as ollama. same endpoints. drop-in replacement for any tool that speaks the openAI completions API.
but here's the difference: pure MLX inference runs 20-30% faster than ollama on the same hardware, because there's no middle layer. no python–C bridge. no llama.cpp abstraction. the computation goes straight to the Apple GPU via Metal. on my M4, a 14B model at 8-bit quantization hits ~45 tokens/sec through MLX vs ~35 tokens/sec through ollama — that's the difference between conversational and waiting.
they run side by side, ollama untouched: mlx-serve gemma-4-26b-4bit brings up an MLX server on port 8342 while ollama stays on 11434.
they coexist. ollama for your cloud models and local fallbacks. MLX for raw speed when you're running something that has an MLX-optimized variant. your agents switch between them like it's nothing.
mlx-swift — native in your apps
this is where it gets insane. mlx-swift (1.7K stars) is a Swift API for MLX. no python. no server process. you embed the model directly in your macOS or iOS app. the model runs in the same process as your UI. same binary. same memory space. Apple's own mlx-swift-examples repo shows this working with LLMs, image generation, and MNIST — all as native Swift apps.
i'm building mac apps — breakpoint, promptvault, clipboard sanitizer. all swift, all native. with mlx-swift, i can add local AI to any of them without shipping a python runtime, without running a separate server, without asking users to install anything. the model is part of the app.
the MLX ecosystem
MLX isn't just one library. it's a complete stack for Apple Silicon machine learning:
- mlx — the core framework (25K+ stars). numpy-like API with automatic differentiation, GPU acceleration via Metal, lazy evaluation, and unified memory optimization [1].
- mlx-lm — LLM inference and serving (4.7K stars). openAI-compatible server, chat completions, streaming. what you use day-to-day.
- mlx-swift — native Swift bindings (1.7K stars). embed MLX directly in iOS/macOS apps with zero python dependency.
- mlx-community on hugging face — 500+ pre-quantized models ready to run. gemma, qwen, phi, llama, mistral — all optimized for Apple Silicon [2].
- mlx-audio — audio generation. TTS, music, sound effects. all on-device.
- mlx-vlm — vision language models. image understanding running locally.
what i run on M4
on my M4 with 128GB unified memory (546 GB/s bandwidth), these are the models that make sense for local Apple Silicon inference:
- hermes-4-14B-8bit — daily driver. fast, smart, tool-calling capable. ~45 tokens/sec through MLX at 8-bit quantization. runs my openclaw agent.
- qwen3-coder-8bit — coding tasks. built for code completion, and it shows on apple silicon inference.
- gemma-4-26b-4bit — general reasoning. 26B parameters at 4-bit quantization fits in 128GB unified memory with room to spare.
and here's the thing — they all run simultaneously. unified memory architecture means i can have ollama serving one model on port 11434 while MLX serves another on 8342, and the M4 doesn't break a sweat. try that on a nvidia setup without buying a second GPU.
why local AI on Apple Silicon matters
i've been running local AI since before it was cool. two years of fighting with python environments, broken CUDA installations, models that need more VRAM than any macbook has, tools that crash your entire agent setup because they decided to hijack your port.
MLX is different because it was designed for the hardware i already own. no buying a second GPU. no renting cloud instances for inference. no python dependency hell. just pip install mlx-lm and you're running at near-native speed on Apple Silicon.
and with mlx-swift, the next step is obvious — embed the model directly in your macOS apps. no server. no runtime. no install step for your users. the model is part of the binary. your mac app has an AI brain and nothing else needs to be running.
the M4 was designed for this. MLX just unlocks it.
sources
- Apple MLX Documentation — official framework reference and architecture overview
- MLX Community on Hugging Face — pre-quantized models for Apple Silicon
- Apple Open Source — MLX Project
- MLX Examples — practical code examples for LLMs, image generation, and more
more where this came from. follow the repo.
Try it
Mochi runs on the same idea — a local model on the Mac you already own, nothing else required.