Running a 30B Coding Model Locally on a Mac Studio
A local setup for Qwen3-Coder-30B on Apple Silicon, using MoE architecture and 4-bit quantization to fit a large coding model into 32GB of unified memory.
QwenCode is my local setup for running Qwen3-Coder-30B-A3B-Instruct on a Mac Studio M2 Max with 32GB of unified memory. The goal was a capable coding assistant that runs entirely on my own hardware, no API, no data leaving the machine. The interesting part is that a 30-billion-parameter model has no business fitting in 32GB of RAM, and yet it does.
How 30B fits in 32GB
Two things make it work. The first is the model's architecture. The A3B in the name signals a mixture-of-experts design, so while the model has 30B parameters in total, only a fraction are active on any given token. That is what keeps inference tractable on consumer hardware instead of demanding a data-center GPU.
The second is quantization. The setup uses the UD-Q4_K_XL 4-bit quant, which brings the whole thing down to about 17GB on disk. On a 32GB machine that leaves genuine headroom for context and the rest of the system, and the notes are explicit that on this hardware the Q4 quant is the sweet spot for balancing quality against fit. The model advertises a 256K token context, extendable further, which for a coding assistant matters because you want it to hold a real chunk of a project in view at once.
Throughput lands around 6 tokens per second. I want to be honest that this is not fast. It is usable for coding help where you read the answer as it streams, and it is the price of running a model this size locally on a Mac rather than renting a GPU. Setting expectations there up front is more useful than pretending local inference is free of tradeoffs.
A thin, reproducible setup
The install intentionally leans on boring, reliable pieces. Ollama goes on via Homebrew, the 17GB model is pulled down, a qwen-coder.sh helper script wraps the launch, and the model loads onto the GPU. Nothing exotic. The recommended sampling settings, temperature 0.7, Top-K 20, Top-P 0.8, and a light repetition penalty, are documented so the behavior is consistent run to run, and the model supports tool calling for more capable workflows.
The takeaway
Running a large coding model locally is less about raw hardware and more about picking the right combination of architecture and quantization for the memory you actually have. A mixture-of-experts model keeps active compute low, and a well-chosen 4-bit quant gets the footprint under your RAM ceiling with room to spare. Accept the honest tradeoff, a few tokens per second in exchange for a private, offline assistant, and a Mac Studio becomes a perfectly reasonable home for a 30B model.
Have something that needs building, or stabilizing?
These notes are the work log. The paid work runs through Moonshine Labs, my product and engineering studio. Tell us what you're building, or grab a call.
Work with us →