What Do You Need to Run Qwen Locally? VRAM by Version
Qwen currently has 8 versions you can deploy locally. The smallest, Qwen3 8B, needs 8GB of VRAM at 4-bit; the largest, Qwen3.8 2.4T A95B, is an MoE model and needs 29GB of VRAM plus about 1,286GB of RAM with MoE offloading. The cheapest device that gets one running is the Tesla V100 16GB, at about $370.
| Version | Released | Context | License | ||
|---|---|---|---|---|---|
| Aug 26, 2026 | 176B | 6B | 256K | Qwen Community License 1.0 | |
| Aug 14, 2026 | 27.8B | 27.8B | 256K | Apache-2.0 | |
| Aug 12, 2026 | 2,446B | 95B | 256K | Qwen3.8-Max License | |
| Apr 22, 2026 | 27.8B | 27.8B | 256K | Apache-2.0 | |
| Apr 16, 2026 | 36B | 3B | 256K | Apache-2.0 | |
| Mar 2, 2026 | 9.7B | 9.7B | 256K | Apache-2.0 | |
| Apr 27, 2025 | 32.8B | 32.8B | 40K | Apache-2.0 | |
| Apr 27, 2025 | 8.2B | 8.2B | 40K | Apache-2.0 |
About Qwen
Qwen is Alibaba's large model family (Alibaba was founded in Hangzhou in 1999). Qwen launched in 2023, and that August Alibaba open-sourced Qwen-7B with commercial use allowed. Since then the family has been mostly open-weight, with sizes ranging from under 1B up to a trillion parameters, in both dense and MoE designs. According to Hugging Face's summer 2026 report, there are 151,448 derivative models based on Qwen, the most of any developer.
The past two years brought several turning points. Qwen3, in April 2025, switched to the Apache 2.0 license and can toggle between thinking and non-thinking modes. Qwen3-Next, that September, moved to a hybrid architecture that combines linear attention (Gated DeltaNet) with gated attention, and Qwen3.5, in February 2026, kept that architecture while becoming natively multimodal. From late 2025, flagship versions stopped getting open weights for a while (Qwen3.7-Max, for example, is available only through an API), while small and mid-size models stayed open source. Qwen3.8 2.4T A95B, in August 2026, brought open flagship weights back, but under a custom license with conditions on commercial use.
- VendorAlibaba
- Versions8
- Parameters8.2B – 2,446B
- Latest versionQwen3.8 Flash Next
How much VRAM do Qwen models need at 2-bit?
2-bit, also called Q2, covers common GGUF files such as Q2_K and UD-Q2_K_XL. They are the smallest and lose the most quality, which makes them a way to get a model running first when VRAM is tight. Each row in the table below is a Qwen version and each column is a context length; each cell is the total VRAM needed to deploy it at Q2 (weights + KV cache + 1.1GB runtime overhead).
| Version | WeightsGB | 32K contextGB | 128K contextGB | 256K contextGB | 1M contextGB |
|---|---|---|---|---|---|
| 78.9GGUF | 80.8 | 83.2 | 86.4 | — | |
| 9.8GGUF | 13 | 19.5 | 28.1 | — | |
| 730.7GGUF | 734.9 | 744.1 | 756.5 | — | |
| 11.8GGUF | 15 | 21.5 | 30.1 | — | |
| 12.3GGUF | 14 | 16.1 | 18.7 | — | |
| 4.1GGUF | 6.2 | 9.5 | 13.8 | — | |
| 12.8GGUF | 22.5 | — | — | — | |
| 3.5GGUF | 9.4 | — | — | — |
How much VRAM do Qwen models need at 4-bit?
4-bit, also called Q4, corresponds to the widely used Q4_K_M files, which balance size and quality and are the default choice for local deployment. Each row in the table below is a Qwen version and each column is a context length; each cell is the total VRAM needed to deploy it at Q4.
| Version | WeightsGB | 32K contextGB | 128K contextGB | 256K contextGB | 1M contextGB |
|---|---|---|---|---|---|
| 111.3GGUF | 113.2 | 115.6 | 118.8 | — | |
| 16.5GGUF | 19.7 | 26.2 | 34.8 | — | |
| 1,310.9GGUF | 1,315.1 | 1,324.3 | 1,336.7 | — | |
| 16.8GGUF | 20 | 26.5 | 35.1 | — | |
| 22.1GGUF | 23.8 | 25.9 | 28.5 | — | |
| 5.7GGUF | 7.8 | 11.1 | 15.4 | — | |
| 19.8GGUF | 29.5 | — | — | — | |
| 5GGUF | 10.9 | — | — | — |
How much VRAM do Qwen models need at 8-bit?
8-bit, also called Q8, corresponds to Q8_0 files, which come closest to the original in quality and are also the largest. Each row in the table below is a Qwen version and each column is a context length; each cell is the total VRAM needed to deploy it at Q8.
| Version | WeightsGB | 32K contextGB | 128K contextGB | 256K contextGB | 1M contextGB |
|---|---|---|---|---|---|
| 188.2GGUF | 190.1 | 192.5 | 195.7 | — | |
| 29GGUF | 32.2 | 38.7 | 47.3 | — | |
| 2,600.2GGUF | 2,604.4 | 2,613.6 | 2,626 | — | |
| 28.6GGUF | 31.8 | 38.3 | 46.9 | — | |
| 36.9GGUF | 38.6 | 40.7 | 43.3 | — | |
| 9.5GGUF | 11.6 | 14.9 | 19.2 | — | |
| 34.8GGUF | 44.5 | — | — | — | |
| 8.7GGUF | 14.6 | — | — | — |
FAQ
Is Qwen open source and free for commercial use?
For most versions, yes. Qwen3 8B, Qwen3 32B, Qwen3.5 9B, Qwen3.6 27B, Qwen3.6 35B A3B and Qwen3.8 27B are all Apache 2.0, so you can use them commercially, fine-tune them and redistribute them, as long as you include the license, keep the copyright notices and mark any files you changed. There are two exceptions, and both attach conditions to commercial use. Qwen3.8 Flash Next uses the Qwen Community License 1.0. If you use it in a commercial product with more than 100 million monthly active users or monthly revenue above the threshold, you must prominently display the model name in the interface. In addition, any company (including its affiliates) that runs a model-as-a-service business, or builds a standalone assistant product focused mainly on AI coding or office work, must apply to Qwen for a separate license before commercial use, regardless of size. Qwen3.8 2.4T A95B uses the Qwen3.8-Max License, which has the same attribution requirement, but its separate-license clause applies only to companies in those two lines of business whose revenue over 12 consecutive months exceeds the threshold. Both licenses state that the separate-license clause doesn't apply if you use the model only internally and don't offer its capabilities to third parties.
Can you run Qwen with Ollama?
Every version except the 2.4T flagship has official tags: qwen3:8b, qwen3:32b, qwen3.5:9b, qwen3.6:27b, qwen3.6:35b-a3b and qwen3.8:27b. The Qwen3.8 Flash Next library has no latest tag, so you have to write the tag out in full, such as qwen3.8-flash-next:125b-a6b-q4_K_M. Ollama names it 125B because it doesn't count the separate 51B of lookup tables; the 176B figure on this site includes them. Qwen3.8 2.4T A95B has neither a local nor a cloud version on Ollama. To run it, use llama.cpp with unsloth's GGUFs, or run the official weights on vLLM.