Written by Jakub Rusinowski · Last updated November 12, 2024
The coding and math powerhouse of the open-weight world. Qwen 2.5 models consistently outperform Llama 3.1 at equivalent sizes in STEM tasks.
| Licence | What it permits | Applies to |
|---|---|---|
Apache-2.0 | Commercial use permitted Commercial use permitted. No usage restrictions beyond attribution. | Qwen 2.5 Coder 32B, Qwen 2.5 14B Instruct, Qwen 2.5 7B Instruct, Qwen 2.5 72B Instruct |
| Qwen 2.5 Coder 32B | Min 20 GB VRAM · Q4_K_M · 128,000 ctx · ollama run qwen2.5-coder:32b |
| Qwen 2.5 14B Instruct | Min 9 GB VRAM · Q4_K_M · 128,000 ctx · ollama run qwen2.5:14b |
| Qwen 2.5 7B Instruct | Min 5 GB VRAM · Q4_K_M · 128,000 ctx · ollama run qwen2.5:7b |
| Qwen 2.5 72B Instruct | Min 44 GB VRAM · Q4_K_M · 128,000 ctx · ollama run qwen2.5:72b |
The cheapest GPU that runs Qwen 2.5 Family locally (min 5 GB VRAM) is the Intel Arc B570 (10 GB).
Install Ollama then run: ollama run qwen2.5-coder:32b
Minimum VRAM: 5 GB. For best results use Q4_K_M quantization.
Pick a quantization and open it in LM Studio, Ollama, or Jan, or download the raw .gguf file directly. Quant list and sizes resolved from Hugging Face.
| Quant | Size | Download (.gguf) |
|---|---|---|
| Q3_K_M | 3.24 GB (est.) | Qwen2.5-7B-Instruct-Q3_K_M.gguf |
| Q4_K_M | 4.59 GB (est.) | Qwen2.5-7B-Instruct-Q4_K_M.gguf |
| Q5_K_M | 5.39 GB (est.) | Qwen2.5-7B-Instruct-Q5_K_M.gguf |
| Q6_K | 6.23 GB (est.) | Qwen2.5-7B-Instruct-Q6_K.gguf |
| Q8_0 | 8.07 GB (est.) | Qwen2.5-7B-Instruct-Q8_0.gguf |
Download in LM Studio: lms get bartowski/Qwen2.5-7B-Instruct-GGUF
Want this model on your phone? You can run it on your desktop with LM Studio and chat from your iPhone or iPad over an encrypted link — see Run LM Studio Models on Your Phone (LM Link).
Qwen 2.5 Family needs about 5 GB VRAM at Q4_K_M quantization for its smallest variant. Variants: Qwen 2.5 Coder 32B (20 GB, Q4_K_M); Qwen 2.5 14B Instruct (9 GB, Q4_K_M); Qwen 2.5 7B Instruct (5 GB, Q4_K_M); Qwen 2.5 72B Instruct (44 GB, Q4_K_M). On Apple Silicon, unified memory counts toward this requirement.
Yes — Qwen 2.5 Family runs on an RTX 4090 (24 GB) and other 24 GB cards such as the RTX 3090. Smaller variants also fit comfortably on 8–16 GB GPUs at Q4_K_M.
Q4_K_M is the best balance of quality and VRAM for Qwen 2.5 Family in most cases. Choose Q8_0 for near-lossless quality if you have spare VRAM, or smaller quants (Q3/Q2) only when memory is tight.
Install Ollama, then run: ollama run qwen2.5-coder:32b. This downloads Qwen 2.5 Family and starts a local, OpenAI-compatible endpoint — no internet connection is needed after the initial download.