Can I Run GPT-OSS on NVIDIA H100 80GB (PCIe)?

作者: Jakub Rusinowski · 最后更新: 2026年8月15日

Yes, comfortably — GPT-OSS 120B at Q4_K_M needs 71.6 GB at 8K context (70.5 GB weights + 1.1 GB KV cache/overhead), leaving ~8.4 GB of the NVIDIA H100 80GB (PCIe)'s 80 GB, at ~240 tok/s (est.), with room for up to 128K context.

See what else this hardware can run →

购买此硬件 NVIDIA H100 80GB PCIe — 80 GB VRAM · 350 W board power立即云端部署 RunPod 上的 NVIDIA H100 80GB — 低至 $2.89/小时 · 价格核实于 2026-07

或在 Vast.ai 比较

作为亚马逊联盟成员,我们从符合条件的购买中获得收入。云 GPU 链接为推荐链接——我们可能获得佣金,您无需额外付费。

联盟营销声明: 本页部分链接为联盟推广链接——如果你通过它们购买,LLM Configurator 可能会获得佣金,而你无需支付任何额外费用。作为亚马逊联盟成员(Amazon Associate),LLM Configurator 会从符合条件的购买中获得收益。

NVIDIA H100 80GB PCIe
80 GB VRAM · 350 W board power
2026年价格波动较大——请以当前商品页价格为准。
在亚马逊查看价格

NVIDIA H100 80GB (PCIe) Specs

VRAM80 GB
Memory Bandwidth2000 GB/s

GPT-OSS 120B on the NVIDIA H100 80GB (PCIe): VRAM by quantization

QuantVRAM neededFits 80 GB?Max contextWhole PC
F16234.7 GB✗ No——
Q8_0125.2 GB✗ No——
Q6_K96.9 GB✗ No——
Q5_K_M83.9 GB✗ No——
Q4_K_M71.6 GB✓ Yes128K—
Q3_K_M50.9 GB✓ Yes128K—
Q2_K39.5 GB✓ Yes128K—

Assumes an 8K-token context with an f16 KV cache. A longer window needs more; a quantized KV cache needs less. “Max context” is the largest window that still fits in 80 GB. Figures are estimates from parameter count, quantization and memory bandwidth — the analyzer lets you tune KV-cache quant and context.

Context costs VRAM too. At Q4_K_M on the NVIDIA H100 80GB (PCIe): 8K 71.6 GB · 32K 72.6 GB · 128K 76.2 GB. Every window this model publishes still fits 80 GB.

No compatible GPU? GPT-OSS 120B on 96 GB of system RAM, CPU only: runs, but it is tight.

GPT-OSS Sizes That Fit the NVIDIA H100 80GB (PCIe) (at 8K context)

GPT-OSS 120BQ4_K_M · 71.6 GB at 8K context · ~240 tok/s (est.)
GPT-OSS 20BMXFP4 · 12.1 GB at 8K context · ~291 tok/s (est.)
Buy vs. rent GPT-OSS
Buy the GPU
~$25,000
NVIDIA H100 80GB (PCIe) · MSRP
Rent by the hour
from $0.77/hr
A100 (80 GB) class

At 2 hrs/day, buying (~$25,000) beats renting at $0.77/hr after about 45.1 years.

Affiliate links — we may earn a commission if you sign up, at no extra cost to you.

Vast.ai $0.77/hr · typical low · varies
Rent on Vast.ai →
RunPod $1.39/hr
Rent on RunPod →

Cloud rates verified 2026-07 — estimates, and marketplace prices vary. Buying price is GPU MSRP only, not a full PC.

FAQ

Is the NVIDIA H100 80GB (PCIe) enough to run GPT-OSS locally?

Yes, comfortably — GPT-OSS 120B at Q4_K_M needs 71.6 GB at 8K context (70.5 GB weights + 1.1 GB KV cache/overhead), leaving ~8.4 GB of the NVIDIA H100 80GB (PCIe)'s 80 GB, at ~240 tok/s (est.), with room for up to 128K context.

What's the best GPT-OSS size for the NVIDIA H100 80GB (PCIe)?

GPT-OSS 120B at Q4_K_M quantization needs 71.6 GB at 8K context (70.5 GB weights + 1.1 GB KV cache/overhead), estimated ~240 tokens/sec, up to 128K context.

Every GPT-OSS size on the NVIDIA H100 80GB (PCIe)

SizeVRAM at 8KVerdictSpeed
GPT-OSS 120B71.6 GBRuns~240 tok/s
GPT-OSS 20B12.1 GBRuns~291 tok/s

GPT-OSS on Other GPUs

Popular Models on the NVIDIA H100 80GB (PCIe)

VRAM Tier

Troubleshooting

Buying Guide

← Can I Run It? | GPT-OSS Model Page | NVIDIA H100 80GB (PCIe) GPU Page | VRAM calculator | Check Your Hardware