Local LLM inference in 2026: Mac Studio vs. RTX 3090 vs. RTX 5090
I benchmarked two Qwen3 architectures — Qwen3-32B (dense) and Qwen3-30B-A3B (a mixture-of-experts, or MoE, model) — on three consumer hardware setups to see how they compare on speed and efficiency. Each of five prompts was run ten times with a cap of 2,048 output tokens, and all metrics (tokens per second among them) were averaged over those runs. To obtain near-deterministic output through greedy decoding, temperature, top_k, top_p, and repeat_penalty were set to 0, 1, 1, and 1, respectively.
The hardware:
-
Apple Mac Studio M4 Max — 128 GB unified memory, ~80–135 W under load
-
NVIDIA RTX 3090 — 24 GB VRAM, ~440–460 W system power draw
-
NVIDIA RTX 5090 — 32 GB VRAM, ~370–610 W system power draw
Both GPU systems share the same configuration:
- CPU: Intel Core i5-14400
- RAM: 64 GB DDR5-4800 (CL40)
- OS: Debian 13 (virtualized)
Model 1 — Qwen3-32B (dense, Q4_K_M, 16K context):
| System | Tokens per Second (TPS) | Power draw (W) | TPS per Watt |
|---|---|---|---|
| Mac Studio | 22 | 135(*) | 0.16 |
| RTX 3090 | 37 | 457 | 0.08 |
| RTX 5090 | 61 | 609 | 0.10 |
Model 2 — Qwen3-30B-A3B (MoE, Q4_K_M, 16K context):
| System | Tokens per Second (TPS) | Power draw (W) | TPS per Watt |
|---|---|---|---|
| Mac Studio | 96 | 82(*) | 1.17 |
| RTX 3090 | 152 | 444 | 0.34 |
| RTX 5090 | 203 | 366 | 0.55 |
(*) Measured at the wall, because powermetrics reports power unreliably. All power figures are whole-system draw.
Interpretation:
-
The Mac Studio is the most efficient of the three systems. On MoE models it delivers roughly 2–3.5 times more tokens per watt than either NVIDIA card. Its 128 GB of unified memory also makes inference on large MoE models (Qwen3.5-122B-A10B at 4-bit, say) and long context windows feasible — something no single consumer GPU can match. Indeed, the two models above fit on the GPUs at all only thanks to 4-bit quantization.
-
It is, however, subject to clear limitations. Dense-model inference is noticeably slower than on the NVIDIA cards, and the same goes for image and video generation (with FLUX models, for instance). Compatibility is worth keeping an eye on too: tooling that requires no configuration on NVIDIA hardware often needs adaptation on Apple silicon.
-
Consumer GPUs are fast, but VRAM-constrained. The 24 GB on the 3090 — and even the 32 GB on the 5090 — place a hard limit on model size and context length unless aggressive quantization is applied. It should further be noted that NVIDIA's GeForce driver license excludes data centre deployment, which may be relevant for production use.
All runs were executed under Ollama without additional tuning, favoring reproducibility over peak performance. Optimized runtimes such as vLLM or MLX are reported to attain higher throughput; this was not measured here.