Every GPU reviewed for local LLM inference, ranked by VRAM and tokens/sec speed on Llama 3.1 8B Q4_K_M — now with the memory bandwidth that figure is derived from.
Specs verified 2026-07-21 · Prices verified 2026-07-06
| Device | Memory | Bandwidth | Llama 3.1 8B Q4_K_M | Price | $/GB | TDP | Released | Runs with |
|---|---|---|---|---|---|---|---|---|
| Apple M3 Ultra | 512 GB unified | 819 GB/s | 43–90 tok/s | $3,999 | $7.81 | 60 W | 2025 | metal |
| Apple M2 Ultra | 192 GB unified | 800 GB/s | 43–89 tok/s | $3,999 | $20.83 | 60 W | 2023 | metal |
| Apple M1 Ultra | 128 GB unified | 800 GB/s | 43–89 tok/s | $3,999 | $31.24 | 60 W | 2022 | metal |
| Apple M4 Max | 128 GB unified | 546 GB/s | 31–64 tok/s | $3,499 | $27.34 | 35 W | 2024 | metal |
| Apple M5 Max | 128 GB unified | 614 GB/s | 34–71 tok/s | $3,499 | $27.34 | 35 W | 2026 | metal |
| Apple M2 Max | 96 GB unified | 400 GB/s | 24–49 tok/s | $3,499 | $36.45 | 40 W | 2023 | metal |
| NVIDIA RTX PRO 6000 Blackwell | 96 GB | 1,792 GB/s | 113–234 tok/s | $8,565 | $89.22 | 600 W | 2025 | cuda, vulkan |
| NVIDIA A100 80GB | 80 GB | 1,935 GB/s | 102–196 tok/s | $15,000 | $187.50 | 300 W | 2021 | cuda, vulkan |
| NVIDIA H100 80GB | 80 GB | 2,000 GB/s | 120–250 tok/s | $25,000 | $312.50 | 350 W | 2022 | cuda, vulkan |
| Apple M1 Max | 64 GB unified | 400 GB/s | 24–49 tok/s | $3,499 | $54.67 | 30 W | 2021 | metal |
| Apple M3 Max | 64 GB unified | 400 GB/s | 24–49 tok/s | $2,499 | $39.05 | 35 W | 2023 | metal |
| Apple M5 Pro | 64 GB unified | 307 GB/s | 19–39 tok/s | $1,999 | $31.23 | 30 W | 2026 | metal |
| NVIDIA L40S | 48 GB | 864 GB/s | 77–147 tok/s | $7,499 | $156.23 | 350 W | 2023 | cuda, vulkan |
| NVIDIA RTX 6000 Ada Generation | 48 GB | 960 GB/s | 83–159 tok/s | $6,799 | $141.65 | 300 W | 2022 | cuda, vulkan |
| Apple M3 Pro | 36 GB unified | 153 GB/s | 9.7–20 tok/s | $1,999 | $55.53 | 30 W | 2023 | metal |
| Apple M1 Pro | 32 GB unified | 200 GB/s | 13–26 tok/s | $1,999 | $62.47 | 30 W | 2021 | metal |
| Apple M2 Pro | 32 GB unified | 200 GB/s | 13–26 tok/s | $1,999 | $62.47 | 30 W | 2023 | metal |
| Apple M4 | 32 GB unified | 120 GB/s | 7.7–16 tok/s | $999 | $31.22 | 22 W | 2024 | metal |
| Apple M5 | 32 GB unified | 153 GB/s | 9.7–20 tok/s | $1,599 | $49.97 | 23 W | 2025 | metal |
| NVIDIA GeForce RTX 5090 | 32 GB | 1,792 GB/s | 113–234 tok/s | $1,999 | $62.47 | 575 W | 2025 | cuda, vulkan |
| Apple M2 | 24 GB unified | 100 GB/s | 6.4–13 tok/s | $1,099 | $45.79 | 20 W | 2022 | metal |
| Apple M3 | 24 GB unified | 100 GB/s | 6.4–13 tok/s | $1,099 | $45.79 | 22 W | 2023 | metal |
| Apple M4 Pro | 24 GB unified | 273 GB/s | 17–35 tok/s | $1,999 | $83.29 | 30 W | 2024 | metal |
| NVIDIA GeForce RTX 3090 | 24 GB | 936 GB/s | 60–115 tok/s | $1,499 | $62.46 | 350 W | 2020 | cuda, vulkan |
| NVIDIA GeForce RTX 4090 | 24 GB | 1,008 GB/s | 86–165 tok/s | $1,599 | $66.63 | 450 W | 2022 | cuda, vulkan |
| AMD Radeon RX 7900 XTX | 24 GB | 960 GB/s | 54–112 tok/s | $999 | $41.63 | 355 W | 2022 | rocm, vulkan |
| AMD Radeon RX 7900 XT | 20 GB | 800 GB/s | 47–98 tok/s | $899 | $44.95 | 315 W | 2022 | rocm, vulkan |
| Apple M1 | 16 GB unified | 68 GB/s | 4.4–9.2 tok/s | $999 | $62.44 | 20 W | 2020 | metal |
| NVIDIA GeForce RTX 4060 Ti 16GB | 16 GB | 288 GB/s | 31–59 tok/s | $499 | $31.19 | 165 W | 2023 | cuda, vulkan |
| NVIDIA GeForce RTX 4070 Ti Super | 16 GB | 672 GB/s | 63–121 tok/s | $799 | $49.94 | 285 W | 2024 | cuda, vulkan |
| NVIDIA GeForce RTX 4080 | 16 GB | 717 GB/s | 66–127 tok/s | $1,199 | $74.94 | 320 W | 2022 | cuda, vulkan |
| NVIDIA GeForce RTX 4080 Super | 16 GB | 736 GB/s | 68–130 tok/s | $999 | $62.44 | 320 W | 2024 | cuda, vulkan |
| NVIDIA GeForce RTX 5060 Ti 16GB | 16 GB | 448 GB/s | 40–83 tok/s | $429 | $26.81 | 180 W | 2025 | cuda, vulkan |
| NVIDIA GeForce RTX 5070 Ti | 16 GB | 896 GB/s | 70–145 tok/s | $749 | $46.81 | 300 W | 2025 | cuda, vulkan |
| NVIDIA GeForce RTX 5080 | 16 GB | 960 GB/s | 74–153 tok/s | $999 | $62.44 | 360 W | 2025 | cuda, vulkan |
| AMD Radeon RX 7800 XT | 16 GB | 624 GB/s | 39–81 tok/s | $499 | $31.19 | 263 W | 2023 | rocm, vulkan |
| AMD Radeon RX 9060 XT 16GB | 16 GB | 320 GB/s | 24–49 tok/s | $349 | $21.81 | 160 W | 2025 | rocm, vulkan |
| AMD Radeon RX 9070 | 16 GB | 640 GB/s | 42–87 tok/s | $549 | $34.31 | 220 W | 2025 | rocm, vulkan |
| AMD Radeon RX 9070 XT | 16 GB | 717 GB/s | 46–95 tok/s | $599 | $37.44 | 220 W | 2025 | rocm, vulkan |
| Intel Arc B580 | 12 GB | 456 GB/s | 28–57 tok/s | $249 | $20.75 | 190 W | 2024 | sycl, openvino, vulkan |
| NVIDIA GeForce RTX 3060 (12GB) | 12 GB | 360 GB/s | 26–50 tok/s | $329 | $27.42 | 170 W | 2021 | cuda, vulkan |
| NVIDIA GeForce RTX 4070 | 12 GB | 504 GB/s | 50–96 tok/s | $599 | $49.92 | 200 W | 2023 | cuda, vulkan |
| NVIDIA GeForce RTX 4070 Super | 12 GB | 504 GB/s | 50–96 tok/s | $599 | $49.92 | 220 W | 2024 | cuda, vulkan |
| NVIDIA GeForce RTX 4070 Ti | 12 GB | 504 GB/s | 50–96 tok/s | $799 | $66.58 | 285 W | 2023 | cuda, vulkan |
| NVIDIA GeForce RTX 5070 | 12 GB | 672 GB/s | 56–116 tok/s | $549 | $45.75 | 250 W | 2025 | cuda, vulkan |
| Intel Arc B570 | 10 GB | 380 GB/s | 24–49 tok/s | $219 | $21.90 | 150 W | 2025 | sycl, openvino, vulkan |
| NVIDIA GeForce RTX 3080 (10GB) | 10 GB | 760 GB/s | 50–97 tok/s | $699 | $69.90 | 320 W | 2020 | cuda, vulkan |
| NVIDIA GeForce RTX 3070 | 8 GB | 448 GB/s | 32–61 tok/s | $499 | $62.38 | 220 W | 2020 | cuda, vulkan |
| NVIDIA GeForce RTX 3070 Ti | 8 GB | 608 GB/s | 42–80 tok/s | $599 | $74.88 | 290 W | 2021 | cuda, vulkan |
| NVIDIA GeForce RTX 4060 | 8 GB | 272 GB/s | 29–56 tok/s | $299 | $37.38 | 115 W | 2023 | cuda, vulkan |
| NVIDIA GeForce RTX 5060 | 8 GB | 448 GB/s | 40–83 tok/s | $299 | $37.38 | 145 W | 2025 | cuda, vulkan |
| NVIDIA GeForce RTX 5060 Ti 8GB | 8 GB | 448 GB/s | 40–83 tok/s | $379 | $47.38 | 180 W | 2025 | cuda, vulkan |
| AMD Radeon RX 9060 XT 8GB | 8 GB | 320 GB/s | 24–49 tok/s | $299 | $37.38 | 150 W | 2025 | rocm, vulkan |
| Under $300 | RTX 3060 12GB — 12 GB VRAM, runs all 7–8B models |
| Under $500 | RTX 4060 Ti 16GB — best budget 16 GB card |
| $500–700 | RTX 4070 Super or AMD RX 9070 XT |
| $1,000–1,600 | RTX 4090 24GB — runs 70B models, fastest single card |
| Mac laptop | M3 Pro 36GB or M4 Pro — silent, power-efficient |
| Mac desktop | M4 Max 128GB — runs any quantized model |
Looking wider? Every device we track · Laptops · AI stations and mini-PCs