GPU comparison · live prices
A100 vs 4090: price per hour and cost per token
Last generation, often the cheapest datacenter GPU per hour for models that fit. The cheapest way to serve 7B–14B models, mostly on marketplaces and spot.
Prices observed 12m ago · refreshed every 15 minutes
| NVIDIA A100 80GB | NVIDIA RTX 4090 | |
|---|---|---|
| Memory | 80 GB HBM2e | 24 GB GDDR6X |
| Memory bandwidth | 2.0 TB/s | 1.0 TB/s |
| Cheapest on-demandper GPU-hour | $0.34Vast.ai | $0.31Vast.ai |
| Cheapest spotper GPU-hour | —None in stock | —None in stock |
| Providers in stock | 3 | 3 |
Per GPU-hour on demand, the 4090 costs 0.93× the A100. Per token, that depends on the model:
Cost per 1M output tokens
Cheapest on-demand offer, GPUs per replica to fit the model, 60% average utilization.
A100 is cheaper per token for every model below that runs on both at today’s prices.
| Model | A100 | 4090 |
|---|---|---|
| Llama 3.1 8BBF16 | $0.05 1× GPUs · ~3,059 tok/s | $0.10 1× GPUs · ~1,512 tok/s |
| Qwen 2.5 32BFP8 | $0.10 1× GPUs · ~1,506 tok/s | $0.32 2× GPUs · ~893 tok/s |
| Llama 3.3 70BFP8 | $0.22 2× GPUs · ~1,386 tok/s | $0.70 4× GPUs · ~822 tok/s |
| Mixtral 8x22BFP8 | $0.12 4× GPUs · ~5,019 tok/s | $0.39 8× GPUs · ~2,977 tok/s |
| Llama 3.1 405BFP8 | $1.29 8× GPUs · ~967 tok/s | — Doesn’t fit on 8 |
| DeepSeek V3 (671B MoE)FP8 | — Doesn’t fit on 8 | — Doesn’t fit on 8 |
Estimates: tokens/s from memory bandwidth with batched serving, not a benchmark. Check each provider before you buy.