Retrieved article excerpt
Open article · Retrieved 2026-09-13T13:21:39.034612+00:00
## TL;DR
Three facts and one command.
**Bench** — Mac mini M4 · 24 GB unified memory · macOS 26.5 · Ollama 0.34.0 · **gemma4:26b** Q4\_K\_M (MoE, 25.2B total / ~4B active) · measured 2026-09-13
Generation speed
11 → 23 tok/s
~2x, same model, same prompt
Processor split (`ollama ps`)
66% CPU → 100% GPU
9 of 31 layers were spilling experts to CPU
The one setting that matters
iogpu.wired\_limit\_mb
macOS default ≈ 17.8 GiB → set to 20480
```
sudo sysctl iogpu.wired_limit_mb=20480 # raise the Metal wired-memory ceiling
# restart Ollama, then:
ollama ps # PROCESSOR must read "100% GPU"
```
**Do not start with the "obvious" tweaks.**
Lowering context or quantizing the KV cache *before* raising the ceiling made things worse here: every request failed with a Metal out-of-memory error. See [the pitfall](https://mac-mini-m4-doc.masterfabric.co/#pitfall).
## Why it is slow out of the box
Unified memory is not the same as "the GPU can use all 24 GB".
macOS limits how much of unified memory the GPU may pin ("wire"). On a 24 GB machine the default ceiling is about **17.8 GiB**. Ollama measures this as its available GPU memory. `gemma4:26b` in Q4\_K\_M projects to about **18.1 GB** on-device once you add a 64K context, the vision projector and the MTP draft model used for speculative decoding. It does not fit, so Ollama's fitter (`common_params_fit_impl`) keeps roughly 3 GB of headroom and moves the mixture-of-experts weights of 9 layers to the CPU.
06 GB12 GB18 GB24 GB
Before
stock ceiling
GPU: 12.7 GB of model
CPU 5.4 GB
Metal ceiling 17.8 GiB
macOS + Cursor
After
wired\_limit 20480
GPU: entire model 18.1 GB (incl. 64K ctx, mmproj, MTP draft)
20 GiB
fit
margin
macOS + Cursor
~3.5 GB
Source: ollama server.log — before: MTL0\_Mapped 8798 MiB + CPU\_Mapped 5376 MiB;
after: "projected to use 18146 MiB of device memory", "no changes needed".
Unified memory, 24 GB. Before: the fitter overflows 5.4 GB of expert weights to CPU to stay under the Metal ceiling. After: the ceiling moves to 20 GiB and the whole model is wired on the GPU.
Why does a CPU spill hurt so much on a MoE model? Generation is memory-bandwidth-bound and, for `gemma4:26b`, only ~4B parameters are active per token, which is why it can stream at dense-4B speeds. But when the expert tensors of 9 layers live on the CPU, every token does a CPU matmul plus a CPU↔GPU sync. You are paying the MoE routing cost without the MoE speed.
## Diagnose your own machine
Two commands and two log lines tell you everything.
```
ollama ps
# NAME SIZE PROCESSOR CONTEXT
# gemma4:26b 4.0 GB 66%/34% CPU/GPU 65536 ← anything but "100% GPU" is the problem
sysctl iogpu.wired_limit_mb
# iogpu.wired_limit_mb: 0 ← 0 = macOS default (~2/3 of RAM on a 24 GB M4)
grep -E "MTL0 \(Apple M4\)|CPU_Mapped|MTL0_Mapped|gpu memory" ~/.ollama/logs/server.log | tail -5
# gpu memory ... library=Metal available="17.3 GiB" free="17.8 GiB"
# load_tensors: MTL0_Mapped model buffer size = 8798.74 MiB
# load_tensors: CPU_Mapped model buffer size = 5376.00 MiB ← expert weights on CPU
# common_params_fit_impl: - MTL0 (Apple M4): 31 layers ( 9 overflowing), 14706 MiB used, 3479 MiB free
```
Before
9 overflowing
After
0 overflowing
31 MoE layers. Hatched = expert weights ("overflow") placed on CPU by Ollama's fitter.
Attention and dense parts of those layers stay on the GPU.
What "9 overflowing" in the log means physically.
## The fix
Raise the ceiling, persist it, then (and only then) tune the rest.
1. ### Raise the Metal wired limit (needs admin)
20 GiB leaves about 3.5–4 GB for macOS and your editor. It is applied instantly and lost on reboot, so also install a LaunchDaemon.
```
sudo sysctl iogpu.wired_limit_mb=20480
sudo tee /Library/LaunchDaemons/com.minipod.iogpu-wired-limit.plist >/dev/null <<'EOF'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>com.minipod.iogpu-wired-limit</string>
<key>ProgramArguments</key><array>
<string>/usr/sbin/sysctl</string><string>iogpu.wired_limit_mb=20480</string>
</array>
<key>RunAtLoad</key><true/>
</dict></plist>
EOF
sudo chown root:wheel /Library/LaunchDaemons/com.minipod.iogpu-wired-limit.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/com.minipod.iogpu-wired-limit.plist
```
2. ### Give Ollama.app its environment via launchd
Ollama.app on macOS does not read `~/.zshrc`. Use `launchctl setenv` from a login-time LaunchAgent. Gate the KV-cache quantization on the ceiling actually being raised (see the pitfall below for why).
```
# ~/mini-pod/scripts/login-env.sh (run by a LaunchAgent at login)
launchctl setenv OLLAMA_KEEP_ALIVE 24h # 26B cold load is ~20 s; keep it resident
launchctl setenv OLLAMA_MAX_LOADED_MODELS 1 # 24 GB: one model at a time
launchctl setenv OLLAMA_NUM_PARALLEL 1 # single user = fastest
launchctl setenv OLLAMA_FLASH_ATTENTION 1
if [[ "$(sysctl -n iogpu.wired_limit_mb)" -ge 20000 ]]; then
launchctl setenv OLLAMA_KV_CACHE_TYPE q8_0 # halves KV memory; never q4_0 (recall degrades)
else
launchctl unsetenv OLLAMA_KV_CACHE_TYPE
fi
pgrep -xq Ollama || open -a Ollama # start Ollama *after* env is set
```
3. ### Set the context length where Ollama.app actually reads it
`OLLAMA_CONTEXT_LENGTH` is silently overridden by the app's own setting stored in SQLite. 32K is enough for agent turns and shortens the worst-case prefill; set it while Ollama is stopped.
```
sqlite3 "$HOME/Library/Application Support/Ollama/db.sqlite" \
"update settings set context_length=32768 where id=1;"
```
4. ### Restart and verify
```
pkill -x Ollama; pkill -f "Ollama.app/Contents/Resources/ollama"; sleep 3; open -a Ollama
curl -s localhost:11434/api/generate -d '{"model":"gemma4:26b","prompt":"hi","stream":false}' >/dev/null
ollama ps # → 100% GPU
grep -c kIOGPUCommandBufferCallbackErrorOutOfMemory ~/.ollama/logs/server.log # → 0
sysctl kern.memorystatus_vm_pressure_level # 1 ok · 2 warn · 4 critical
```
**Scripted version**
The repo this page ships with wraps steps 1–4 as `sudo ./scripts/install-gpu-limit.sh` (root half) and `./scripts/apply-ollama-tuning.sh` (user half, with a built-in smoke test). `./scripts/status.sh` prints every value on this page's checklist.
## The pitfall: tuning the "safe" knobs first breaks the model
We tried q8\_0 KV cache and a 32K context before touching the ceiling. Every request then failed.
Intuition says a smaller context and a smaller KV cache can only help. In practice the fitter saw the freed memory and kept *more* expert layers on the GPU (9 → 6 overflowing). Its projection does not fully account for Metal heap overhead, the vision projector and the MTP draft model, so at decode time the GPU ran out of wired memory:
```
ggml_metal_synchronize: error: command buffer 0 failed with status 5
error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
ggml_metal_graph_compute: backend is in error state from a previous command buffer failure
srv update_slots: decode() failed: Compute error.
```
stock ceiling 17.8 GiB
raised 20 GiB
64K · f16 KV · stock
3.4 GB margin
slow, works
32K · q8\_0 KV · stock
OOM
Compute error
32K · q8\_0 KV · 20480
3.5 GB margin
fast, works
Blue = what the fitter places on the GPU. The runtime peak (Metal heap, mmproj, MTP draft) sits above the fitter's
projection; with only a 27 MiB paper margin (middle row) it crosses the stock ceiling and the backend faults.
Same model, three configurations. The middle row is the trap: a smaller context makes the fitter greedier, and the stock ceiling has no room for the runtime overshoot.
**Rule**
With the stock ceiling, the only stable configuration we found was the slow one (64K, f16). Change the ceiling first; add q8\_0 and a smaller context second. If you ever see `kIOGPUCommandBufferCallbackErrorOutOfMemory`, unload the model (`curl localhost:11434/api/generate -d '{"model":"gemma4:26b","keep_alive":0}'`) — the Metal backend stays in an error state until the runner is recreated.
## Results
Same machine, same model file, same prompts, `/api/generate` timings.
Generation (tok/s)
64–256 token outputs
11.0 before
23.0 after (2.1x)
Prefill (tok/s)
6.7K–36K token prompts
255 before
234 after (unchanged)
Short request, end-to-end (s)
23-tok prompt, 64-tok answer
6.0 before
3.0 after (2x)
Bars are scaled per row. Prefill is compute-bound and the spilled experts were already batch-efficient on the CPU,
so it does not move; token streaming, which is bandwidth-bound, does.
Generation doubles, latency halves, prefill stays put.
| Metric | Before (66% CPU) | After (100% GPU) | Notes |
| --- | --- | --- | --- |
| Generation, short prompt | 9.5 – 12.5 tok/s | 22.5 – 23.5 tok/s | ~2x |
| Generation, 256-token output | — | 22.2 tok/s | steady, no decay |
| Prefill, 6.7K-token prompt | ~255 tok/s | 234 tok/s | within noise |
| Cold load | ~20 s | ~20 s | mmap; keep-alive hides it |
| Metal OOM events | 0 | 0 | after the ceiling raise |
| Wired memory | ~17.5 GB | ~20.5 GB | pressure level 2 (warn) |
| Swap in use | ~2.0 GB | ~2.3 GB | editor + macOS pay for it |
What you should not expect: cloud-class speed. M4 (non-Pro) has roughly 120 GB/s of memory bandwidth, and generation on a 4B-active MoE tops out around 35–40 tok/s on that budget. 23 tok/s with a 32K context, flash attention and speculative decoding is a realistic place to land on a 24 GB machine that is also running an editor.
## Surviving a reboot
Three things have to happen in order, and nothing does it for you by default.
system · LaunchDaemon
sysctl iogpu.wired\_limit\_mb=20480
runs at boot, before login
gui · LaunchAgent
login-env.sh → launchctl setenv …
reads the sysctl, picks q8\_0 or f16
open -a Ollama
inherits env, measures 20 GiB
app's own login item left disabled
If Ollama starts before the env is set it silently runs with defaults. Making the agent the thing that launches Ollama removes the race.
Two details we tripped on: Ollama.app registers an "open at login" item via SMAppService, but on this machine it was disabled, so nothing started Ollama after a reboot at all. And two copies of the same LaunchAgent had accumulated; harmless, but delete duplicates so there is one source of truth.
Post-reboot check, in one command: `./scripts/status.sh` — it prints the wired limit, the daemon's last exit code, the KV type, the app's context length, the fitter's overflow line, the Metal OOM count and the memory pressure level.
## FAQ / edge cases
### Memory pressure shows level 2 (warn) after the change. Is that ok?
Yes if the box is a dedicated inference node. macOS + an editor have ~3.5 GB. If it reaches level 4 (critical) or the machine starts swapping hard, drop the ceiling one notch: `sudo sysctl iogpu.wired_limit_mb=19456`. The fitter will overflow one layer and you keep most of the speed.
### Still seeing 1–2 overflowing layers at 20480?
Ollama wants roughly 3.1–3.5 GB of headroom. If your model blob or context is a bit larger than ours, go to `21504`. Do not go higher on 24 GB.
### Why not `OLLAMA_NUM_GPU=99` like the older guides say?
Ollama 0.34 already offloads every layer it can (`offloaded 43/43 layers to GPU` in the log). The spill is inside the layers (expert tensors), decided by the fitter, and driven by the memory ceiling. Layer count is not the lever.
### Does `OLLAMA_KV_CACHE_TYPE=q4_0` buy more?
A few hundred MB, at a real cost in long-context recall. Attention re-reads the cache thousands of times; keep it at q8\_0.
### Does `OLLAMA_CO