The Key Driver of LLM Speed Is Memory Bandwidth
When running large language models locally on Apple Silicon chips, the main bottleneck is not the compute power of the GPU or NPU, but the speed of reading model weights from unified memory. To generate each token, the chip must pass the entire array of neural network parameters through memory.
Practical tests on the base M4 processor with 120 GB/s bandwidth and Q4_K_M quantization (using Ollama 0.31.2) showed that real-world generation speed consistently reaches about 83% of the theoretical maximum. This makes it possible to accurately forecast the performance of unreleased hardware configurations before physical benchmarking.
| Model | Size in Q4_K_M, GB | Theoretical Limit (120 GB/s) | Actual Result on M4 | Efficiency Ratio |
|---|---|---|---|---|
| Llama 3.2 3B | 2.0 | 60 tok/s | 46.7 tok/s | 0.78 |
| Mistral 7B | 4.4 | 27 tok/s | 22.8 tok/s | 0.84 |
| Llama 3.1 8B | 4.9 | 24 tok/s | 21.2 tok/s | 0.87 |
| Qwen 2.5 14B | 8.5 | 14 tok/s | 11.7 tok/s | 0.83 |
Memory Subsystem Specs of the M6 and M5 Lineups
When choosing a configuration, it is essential to consider the hardware limitations of entry-level variants. In lower-tier chips, Apple engineers cut memory channels, directly reducing inference speed:
- M6 (16 GB): The cut-down interface delivers 153 GB/s instead of the 170 GB/s found in 24 GB and 32 GB versions (~11% speed loss).
- M5 Max (36 GB): The 32-core GPU version is limited to 460 GB/s, whereas 40-core configurations (48 GB and up) offer 614 GB/s (a 25% difference).
| Processor | RAM Capacity | Memory Bandwidth |
|---|---|---|
| M4 (baseline for comparison) | 16 GB | 120 GB/s |
| M6 | 16 GB | 153 GB/s |
| M6 | 24 / 32 GB | 170 GB/s |
| M5 Pro | up to 64 GB | 307 GB/s |
| M5 Max (32 GPU) | 36 GB | 460 GB/s |
| M5 Max (40 GPU) | 48 / 64 / 128 GB | 614 GB/s |
| M5 Ultra | up to 512 GB | 1.2 TB/s |
Summary Forecast: Tokens per Second Across Models
The calculation is based on an efficiency factor of 0.83 and a 20% memory reserve for system needs and the context buffer (KV-cache). A dash indicates that the model physically does not fit into the available RAM capacity.
| Neural Network (Q4_K_M quant) | Size, GB | M4 (16 GB) | M6 (16 GB) | M6 (32 GB) | M5 Pro (64 GB) | M5 Max (128 GB) | M5 Ultra (512 GB) |
|---|---|---|---|---|---|---|---|
| Llama 3.2 3B | 2.0 | 46.7 (actual) | 64 | 71 | 127 | 255 | 498 |
| Llama 3.1 8B | 4.9 | 21.2 (actual) | 26 | 29 | 52 | 104 | 203 |
| Qwen 2.5 14B | 8.5 | 11.7 (actual) | 15 | 17 | 30 | 60 | 117 |
| Qwen 2.5 32B | 19.0 | — | — | 7.4 | 13 | 27 | 52 |
| Llama 3.3 70B | 40.0 | — | — | — | 6.4 | 13 | 25 |
| Qwen 2.5 72B | 41.0 | — | — | — | 6.2 | 12 | 24 |
| DeepSeek V3 671B | 380.0 | — | — | — | — | — | 2.6 |
Practical Takeaways for Choosing a Device
- The M6 chip does not offer a qualitative leap. A 25–40% increase over the M4 keeps only lightweight 8B models fully comfortable. 14B-class models run right at the threshold of comfortable reading speed.
- M5 Pro is the entry point for 32B models. Reaching 13 tok/s on Qwen 2.5 32B provides a comfortable interactive chat experience, while 8B models hit over 50 tok/s. The 64 GB version also enables running 70B models in the background (around 6.4 tok/s).
- M5 Max unlocks the full potential of 70B models. A speed of 13 tok/s on 70B–72B models makes this chip optimal for complex workloads and multi-agent queries.
- M5 Ultra is essential for ultra-heavy models. It is the only consumer platform capable of fitting DeepSeek V3 (671B) entirely in memory, albeit at a modest speed of about 2.6 tok/s.
Where the Theoretical Model May Diverge from Reality
The mathematical formula may experience discrepancies in edge cases:
- Lightweight models on powerful chips: Llama 3B speed on the M5 Ultra is unlikely to reach 498 tok/s, as the bottleneck shifts from memory bandwidth to engine latency and compute units.
- Ultra interconnect architecture: The M5 Ultra consists of two dies connected via an UltraFusion bridge. Distributing massive model weights across dies may introduce additional overhead.
- Prefill Phase: The formula applies strictly to the token generation (decoding) phase. Processing the input prompt depends on raw GPU compute power and framework optimizations.