Which Open Models Can You Run Locally? A Memory Guide for Qwen, DeepSeek, GLM and gpt-oss
How much memory each popular open-weight model needs at 4-bit, which mini PCs and Macs can hold it, how fast it can go, and when running it yourself beats paying an API per token.
Most of the open models people ask us about will run on a single desktop machine, as long as it has enough memory. The rule of thumb: at 4-bit quantization, a model needs about 0.6GB of memory per billion parameters, plus a few GB for the context. That puts 30B-class models like Qwen3.6 35B A3B on a 32GB machine, 120B-class models like gpt-oss-120b on a 128GB mini PC, and DeepSeek V3.2 only on a 512GB Mac Studio.
Below: what each model needs, which machines fit it, how fast it can go, and when it’s cheaper to just pay an API. Model sizes are from each model’s page; machine prices are from the makers’ stores on 22 November 2026. If you’re choosing hardware, see our mini PC guide for local LLMs.
How much memory does each model need?
Total parameters decide whether a model fits. Active parameters (for mixture-of-experts models, the “A” number) decide how fast it runs. Memory figures are for 4-bit weights; add 2–10GB for context, more for very long prompts.
| Model | Total / active parameters | Memory at 4-bit | Smallest machine that fits |
|---|---|---|---|
| gpt-oss-20b | 21B / 3.6B | ~13GB | Any 24–32GB machine |
| Qwen3.8 27B | 27B dense | ~16GB | 32GB |
| Gemma 4 26B A4B | 26B / 4B | ~16GB | 32GB |
| Qwen3 Coder 30B A3B | 30B / 3B | ~18GB | 32GB |
| Qwen3.6 35B A3B, Qwen3.5 35B A3B | 35B / 3B | ~21GB | 32GB |
| Llama 3.3 70B | 70B dense | ~42GB | 64GB (slow, see below) |
| Qwen3 Next 80B A3B | 80B / 3B | ~48GB | 64GB |
| GLM 4.5 Air | 106B / 12B | ~64GB | 96GB+ (a 128GB mini PC) |
| Llama 4 Scout | 109B / 17B | ~65GB | 96GB+ |
| gpt-oss-120b | 117B / 5.1B | ~63GB (ships in 4-bit) | 96GB+ |
| Mistral Small 4 | 119B | ~71GB | 96GB+ |
| Qwen3.5 122B A10B | 122B / 10B | ~73GB | 96GB+ |
| MiniMax M2.1 | 230B / 10B | ~138GB | 192GB (160GB as VRAM) |
| Qwen3 235B A22B Thinking | 235B / 22B | ~141GB | 192GB |
| Qwen3.5 397B A17B | 397B / 17B | ~238GB | Mac Studio M5 Ultra, 256GB+ |
| DeepSeek V3.2 | 671B / 37B | ~400GB | Mac Studio M5 Ultra, 512GB |
| Kimi K2.6 | 1T / 32B | ~600GB | None of these at 4-bit |
A note on the “96GB+” rows: AMD Ryzen AI Max+ 395 machines have 128GB, of which AMD says up to 96GB can be given to the GPU. That’s enough for the 64–73GB models above with room for context. The newer Ryzen AI Max+ PRO 495 boxes have 192GB, with up to 160GB as VRAM.
The newest flagships, like Qwen3.8 2.4T A95B, GLM-5.3 and Kimi K3, are far too big for any desktop. Use an API for those.
Which machine for which model?
| Memory | Machines (from) | Good fits |
|---|---|---|
| 32GB | Mac mini M6 ($899, up to 32GB), budget Ryzen AI 9 HX 370 mini PCs (~$1,150) | gpt-oss-20b, Qwen3.8 27B, Qwen3.6 35B A3B, Gemma 4 26B |
| 64GB | Mac mini M5 Pro 64GB ($2,699), GMKtec EVO-X2 64GB ($2,199.99), GMKtec EVO-X1 Pro ($1,599.99) | Everything above with longer context, Qwen3 Next 80B A3B |
| 128GB | GMKtec EVO-X2 ($3,499.99), Minisforum MS-S1 Max ($3,959), Beelink GTR9 Pro ($4,199), Mac Studio M5 Max ($5,099), NVIDIA DGX Spark ($6,950) | gpt-oss-120b, Qwen3.5 122B A10B, GLM 4.5 Air, Llama 4 Scout |
| 192GB | GMKtec EVO-X5 Pro ($6,799), Beelink GTR9 Pro 495 ($7,399) | MiniMax M2.1, Qwen3 235B A22B |
| 256–512GB | Mac Studio M5 Ultra (from $5,499 with 96GB; more memory costs extra) | Qwen3.5 397B, DeepSeek V3.2 |
How fast will it run?
Generating each token means reading the model’s active weights from memory once, so memory bandwidth divided by the size of the active weights gives a hard ceiling on tokens per second. Real speeds land well below it, often around half, but the ceiling is a reliable way to compare.
| Model (4-bit) | Active weights | Ceiling at 256 GB/s (Ryzen AI Max+ 395) | at 273 GB/s (DGX Spark) | at 614 GB/s (M5 Max) |
|---|---|---|---|---|
| Qwen3.6 35B A3B | ~1.8GB | ~140 tok/s | ~150 tok/s | ~340 tok/s |
| gpt-oss-120b | ~2.7GB | ~95 tok/s | ~100 tok/s | ~225 tok/s |
| Qwen3.5 122B A10B | ~6GB | ~43 tok/s | ~45 tok/s | ~100 tok/s |
| GLM 4.5 Air | ~7.2GB | ~36 tok/s | ~38 tok/s | ~85 tok/s |
| Qwen3.8 27B (dense) | ~16GB | ~16 tok/s | ~17 tok/s | ~38 tok/s |
| Llama 3.3 70B (dense) | ~42GB | ~6 tok/s | ~6.5 tok/s | ~15 tok/s |
Two things follow. First, mixture-of-experts models are the sweet spot for these machines: gpt-oss-120b is a much bigger model than Qwen3.8 27B but reads far fewer bytes per token, so it’s faster. Second, dense 70B models are painful on anything except a high-bandwidth Mac. For dense models, a rented GPU is much quicker: an RTX 4090 reads memory at about 1 TB/s and an RTX 5090 at about 1.8 TB/s, but they only hold 24GB and 32GB.
Prompt processing (reading a long input before the first token) depends on compute rather than bandwidth. The DGX Spark and the RTX cards are much faster at it than the AMD and Apple machines, which matters for coding agents that send big prompts.
Is running locally cheaper than an API?
Usually not for the popular open models, because API prices for them are very low. Some numbers from our model price comparison:
- gpt-oss-120b costs from $0.03 per million input tokens and $0.092 per million output tokens. A 128GB mini PC generating 50 tokens a second around the clock produces about 130 million tokens a month. The same output from the cheapest API costs about $12. At $3,500 for the machine, that’s over 20 years to break even, before electricity.
- Qwen3.5 122B A10B costs from $0.24 in / $1.92 out. At a realistic 25 tokens a second, 24/7, that’s about 65 million output tokens a month, or about $125 of API output. Break-even on a $3,500 machine is roughly 28 months of nonstop use.
- Renting instead: an RTX 4090 starts at $0.27 an hour on Vast.ai and an RTX 5090 at $0.37, about $200 and $270 a month if you leave them running. That’s fast for models that fit in 24–32GB, but the 120B class won’t fit on one card.
So buy a local machine for the other reasons: privacy (prompts never leave your desk), no rate limits or per-token anxiety while you experiment, working offline, fine-tuning and testing small models, or a workload that really does run all day on a model with pricier APIs. If you only need a few million tokens a month, an API is cheaper: compare providers on each model page above, or start with the free API tiers.
Getting started
The quickest route is LM Studio or Ollama: install, search for the model, and pick a 4-bit (Q4_K_M or MXFP4) file. Both serve an OpenAI-compatible API on localhost, so code written against a hosted provider works with a one-line change of base URL. llama.cpp gives the most control on AMD and Apple machines, and vLLM is the usual choice on NVIDIA hardware like the DGX Spark.
Check the licence before using a model commercially. Most of the Qwen, DeepSeek, GLM and gpt-oss models above are Apache 2.0 or MIT; see the licence table in our Chinese open models guide.
The mini PCs in this guide
GMKtec EVO-X2
- Chip
- AMD Ryzen AI Max+ 395
- Memory
- 64GB or 128GB unified (up to 96GB as VRAM)
- Bandwidth
- 256 GB/s
- Runs
- 128GB: gpt-oss-120b, Qwen3.5 122B-A10B, GLM-4.5 Air. 64GB: 30B-class models
128GB / 1TB is $3,499.99. The cheapest 128GB Ryzen AI Max box we found.
Minisforum MS-S1 Max
- Chip
- AMD Ryzen AI Max+ 395
- Memory
- 128GB unified (up to 96GB as VRAM)
- Bandwidth
- 256 GB/s
- Runs
- gpt-oss-120b, Qwen3.5 122B-A10B, Llama 4 Scout, Mistral Small 4
Sale price (list $4,949) for 128GB / 2TB. Built as a small workstation, with more ports than most.
GMKtec EVO-X5 Pro
- Chip
- AMD Ryzen AI Max+ PRO 495
- Memory
- 192GB unified (up to 160GB as VRAM)
- Bandwidth
- 273 GB/s
- Runs
- Qwen3 235B-A22B, MiniMax M2.1, plus everything the 128GB boxes run
AMD's 2026 chip. 192GB / 2TB; 4TB is $7,099.
NVIDIA DGX Spark
- Chip
- NVIDIA GB10 Grace Blackwell
- Memory
- 128GB unified
- Bandwidth
- 273 GB/s
- Runs
- gpt-oss-120b, Qwen3.5 122B-A10B, Llama 4 Scout; CUDA, so most training tools work
128GB / 4TB Founders Edition. Partner GB10 boxes from ASUS, MSI and Dell cost $5,599 to $8,500.
Mac mini (M5 Pro)
- Chip
- Apple M5 Pro
- Memory
- 24GB or 64GB unified
- Bandwidth
- 307 GB/s
- Runs
- 64GB: Qwen3.8 27B, Qwen3.6 35B-A3B, gpt-oss-20b, Gemma 4 26B
The 64GB model is $2,699; the 24GB base only fits small models.
Mac Studio (M5 Max)
- Chip
- Apple M5 Max
- Memory
- 48GB or 128GB unified
- Bandwidth
- 614 GB/s (40-core GPU)
- Runs
- 128GB: gpt-oss-120b, Qwen3.5 122B-A10B, at more than twice the bandwidth of the AMD boxes
128GB is $5,099. The M5 Ultra (96GB to 512GB, 1.2 TB/s) starts at $5,499.
Starting prices from each maker's own store on November 22, 2026; they change often, so check the store. "Runs" is a rough guide for 4-bit quantized models.