by Inference Hub

Which Open Models Can You Run Locally? A Memory Guide for Qwen, DeepSeek, GLM and gpt-oss

How much memory each popular open-weight model needs at 4-bit, which mini PCs and Macs can hold it, how fast it can go, and when running it yourself beats paying an API per token.

local-inferenceopen-sourceqwendeepseekhardware

Most of the open models people ask us about will run on a single desktop machine, as long as it has enough memory. The rule of thumb: at 4-bit quantization, a model needs about 0.6GB of memory per billion parameters, plus a few GB for the context. That puts 30B-class models like Qwen3.6 35B A3B on a 32GB machine, 120B-class models like gpt-oss-120b on a 128GB mini PC, and DeepSeek V3.2 only on a 512GB Mac Studio.

Below: what each model needs, which machines fit it, how fast it can go, and when it’s cheaper to just pay an API. Model sizes are from each model’s page; machine prices are from the makers’ stores on 22 November 2026. If you’re choosing hardware, see our mini PC guide for local LLMs.

How much memory does each model need?

Total parameters decide whether a model fits. Active parameters (for mixture-of-experts models, the “A” number) decide how fast it runs. Memory figures are for 4-bit weights; add 2–10GB for context, more for very long prompts.

ModelTotal / active parametersMemory at 4-bitSmallest machine that fits
gpt-oss-20b21B / 3.6B~13GBAny 24–32GB machine
Qwen3.8 27B27B dense~16GB32GB
Gemma 4 26B A4B26B / 4B~16GB32GB
Qwen3 Coder 30B A3B30B / 3B~18GB32GB
Qwen3.6 35B A3B, Qwen3.5 35B A3B35B / 3B~21GB32GB
Llama 3.3 70B70B dense~42GB64GB (slow, see below)
Qwen3 Next 80B A3B80B / 3B~48GB64GB
GLM 4.5 Air106B / 12B~64GB96GB+ (a 128GB mini PC)
Llama 4 Scout109B / 17B~65GB96GB+
gpt-oss-120b117B / 5.1B~63GB (ships in 4-bit)96GB+
Mistral Small 4119B~71GB96GB+
Qwen3.5 122B A10B122B / 10B~73GB96GB+
MiniMax M2.1230B / 10B~138GB192GB (160GB as VRAM)
Qwen3 235B A22B Thinking235B / 22B~141GB192GB
Qwen3.5 397B A17B397B / 17B~238GBMac Studio M5 Ultra, 256GB+
DeepSeek V3.2671B / 37B~400GBMac Studio M5 Ultra, 512GB
Kimi K2.61T / 32B~600GBNone of these at 4-bit

A note on the “96GB+” rows: AMD Ryzen AI Max+ 395 machines have 128GB, of which AMD says up to 96GB can be given to the GPU. That’s enough for the 64–73GB models above with room for context. The newer Ryzen AI Max+ PRO 495 boxes have 192GB, with up to 160GB as VRAM.

The newest flagships, like Qwen3.8 2.4T A95B, GLM-5.3 and Kimi K3, are far too big for any desktop. Use an API for those.

Which machine for which model?

MemoryMachines (from)Good fits
32GBMac mini M6 ($899, up to 32GB), budget Ryzen AI 9 HX 370 mini PCs (~$1,150)gpt-oss-20b, Qwen3.8 27B, Qwen3.6 35B A3B, Gemma 4 26B
64GBMac mini M5 Pro 64GB ($2,699), GMKtec EVO-X2 64GB ($2,199.99), GMKtec EVO-X1 Pro ($1,599.99)Everything above with longer context, Qwen3 Next 80B A3B
128GBGMKtec EVO-X2 ($3,499.99), Minisforum MS-S1 Max ($3,959), Beelink GTR9 Pro ($4,199), Mac Studio M5 Max ($5,099), NVIDIA DGX Spark ($6,950)gpt-oss-120b, Qwen3.5 122B A10B, GLM 4.5 Air, Llama 4 Scout
192GBGMKtec EVO-X5 Pro ($6,799), Beelink GTR9 Pro 495 ($7,399)MiniMax M2.1, Qwen3 235B A22B
256–512GBMac Studio M5 Ultra (from $5,499 with 96GB; more memory costs extra)Qwen3.5 397B, DeepSeek V3.2

How fast will it run?

Generating each token means reading the model’s active weights from memory once, so memory bandwidth divided by the size of the active weights gives a hard ceiling on tokens per second. Real speeds land well below it, often around half, but the ceiling is a reliable way to compare.

Model (4-bit)Active weightsCeiling at 256 GB/s (Ryzen AI Max+ 395)at 273 GB/s (DGX Spark)at 614 GB/s (M5 Max)
Qwen3.6 35B A3B~1.8GB~140 tok/s~150 tok/s~340 tok/s
gpt-oss-120b~2.7GB~95 tok/s~100 tok/s~225 tok/s
Qwen3.5 122B A10B~6GB~43 tok/s~45 tok/s~100 tok/s
GLM 4.5 Air~7.2GB~36 tok/s~38 tok/s~85 tok/s
Qwen3.8 27B (dense)~16GB~16 tok/s~17 tok/s~38 tok/s
Llama 3.3 70B (dense)~42GB~6 tok/s~6.5 tok/s~15 tok/s

Two things follow. First, mixture-of-experts models are the sweet spot for these machines: gpt-oss-120b is a much bigger model than Qwen3.8 27B but reads far fewer bytes per token, so it’s faster. Second, dense 70B models are painful on anything except a high-bandwidth Mac. For dense models, a rented GPU is much quicker: an RTX 4090 reads memory at about 1 TB/s and an RTX 5090 at about 1.8 TB/s, but they only hold 24GB and 32GB.

Prompt processing (reading a long input before the first token) depends on compute rather than bandwidth. The DGX Spark and the RTX cards are much faster at it than the AMD and Apple machines, which matters for coding agents that send big prompts.

Is running locally cheaper than an API?

Usually not for the popular open models, because API prices for them are very low. Some numbers from our model price comparison:

  • gpt-oss-120b costs from $0.03 per million input tokens and $0.092 per million output tokens. A 128GB mini PC generating 50 tokens a second around the clock produces about 130 million tokens a month. The same output from the cheapest API costs about $12. At $3,500 for the machine, that’s over 20 years to break even, before electricity.
  • Qwen3.5 122B A10B costs from $0.24 in / $1.92 out. At a realistic 25 tokens a second, 24/7, that’s about 65 million output tokens a month, or about $125 of API output. Break-even on a $3,500 machine is roughly 28 months of nonstop use.
  • Renting instead: an RTX 4090 starts at $0.27 an hour on Vast.ai and an RTX 5090 at $0.37, about $200 and $270 a month if you leave them running. That’s fast for models that fit in 24–32GB, but the 120B class won’t fit on one card.

So buy a local machine for the other reasons: privacy (prompts never leave your desk), no rate limits or per-token anxiety while you experiment, working offline, fine-tuning and testing small models, or a workload that really does run all day on a model with pricier APIs. If you only need a few million tokens a month, an API is cheaper: compare providers on each model page above, or start with the free API tiers.

Getting started

The quickest route is LM Studio or Ollama: install, search for the model, and pick a 4-bit (Q4_K_M or MXFP4) file. Both serve an OpenAI-compatible API on localhost, so code written against a hosted provider works with a one-line change of base URL. llama.cpp gives the most control on AMD and Apple machines, and vLLM is the usual choice on NVIDIA hardware like the DGX Spark.

Check the licence before using a model commercially. Most of the Qwen, DeepSeek, GLM and gpt-oss models above are Apache 2.0 or MIT; see the licence table in our Chinese open models guide.

The mini PCs in this guide

GMKtecfrom $2,199.99

GMKtec EVO-X2

Chip
AMD Ryzen AI Max+ 395
Memory
64GB or 128GB unified (up to 96GB as VRAM)
Bandwidth
256 GB/s
Runs
128GB: gpt-oss-120b, Qwen3.5 122B-A10B, GLM-4.5 Air. 64GB: 30B-class models

128GB / 1TB is $3,499.99. The cheapest 128GB Ryzen AI Max box we found.

Minisforumfrom $3,959

Minisforum MS-S1 Max

Chip
AMD Ryzen AI Max+ 395
Memory
128GB unified (up to 96GB as VRAM)
Bandwidth
256 GB/s
Runs
gpt-oss-120b, Qwen3.5 122B-A10B, Llama 4 Scout, Mistral Small 4

Sale price (list $4,949) for 128GB / 2TB. Built as a small workstation, with more ports than most.

GMKtecfrom $6,799

GMKtec EVO-X5 Pro

Chip
AMD Ryzen AI Max+ PRO 495
Memory
192GB unified (up to 160GB as VRAM)
Bandwidth
273 GB/s
Runs
Qwen3 235B-A22B, MiniMax M2.1, plus everything the 128GB boxes run

AMD's 2026 chip. 192GB / 2TB; 4TB is $7,099.

NVIDIAfrom $6,950

NVIDIA DGX Spark

Chip
NVIDIA GB10 Grace Blackwell
Memory
128GB unified
Bandwidth
273 GB/s
Runs
gpt-oss-120b, Qwen3.5 122B-A10B, Llama 4 Scout; CUDA, so most training tools work

128GB / 4TB Founders Edition. Partner GB10 boxes from ASUS, MSI and Dell cost $5,599 to $8,500.

Applefrom $1,699

Mac mini (M5 Pro)

Chip
Apple M5 Pro
Memory
24GB or 64GB unified
Bandwidth
307 GB/s
Runs
64GB: Qwen3.8 27B, Qwen3.6 35B-A3B, gpt-oss-20b, Gemma 4 26B

The 64GB model is $2,699; the 24GB base only fits small models.

Applefrom $3,099

Mac Studio (M5 Max)

Chip
Apple M5 Max
Memory
48GB or 128GB unified
Bandwidth
614 GB/s (40-core GPU)
Runs
128GB: gpt-oss-120b, Qwen3.5 122B-A10B, at more than twice the bandwidth of the AMD boxes

128GB is $5,099. The M5 Ultra (96GB to 512GB, 1.2 TB/s) starts at $5,499.

Starting prices from each maker's own store on November 22, 2026; they change often, so check the store. "Runs" is a rough guide for 4-bit quantized models.

Subscribe to our newsletter

News and updates, 1–2 emails per month. No spam.