Overview
Every saving and every risk across your inference stack, ranked by what it’s worth per month.
GPU spend
$46k
19 GPUs across 4 providers
Paid for, not used
$20k
44% of spend was idle
Cost per 1M tokens
$0.361
Blended, all endpoints
Savings found
$20k/mo
4 changes, 2 risks
October so far
Day 23 of 31. The projection extends the last 7 days’ average to month end; it’s an estimate.
Spent this month
$35,310
Projected month end
$47,532
at $1,528/day
Budget
$45,000
$2,532 over at this pace
Ranked opportunities
Sorted by monthly value. Risks have no price, so they sit below savings. Download CSV
- 1 $7,390/mo
Move doc-summarize off AWS H100 on-demand
Same 4× H100 on RunPod spot at $1.89/hr vs $4.42/hr. Nightly summaries tolerate preemption.
- 2 $6,110/mo
Right-size support-agent from 8× to 5× H100
Load test holds p95 at 980 ms on 5 GPUs at current peak. Three GPUs sit idle most of the day.
- 3 $5,510/mo
Move transcribe from 2× H100 to 1× A100 80GB
Whisper needs under 5 GB and runs at 22% utilization. One A100 covers peak with headroom.
- 4 $1,370/mo
Scale sql-copilot to 2× H200 on weekends
Analysts barely query on weekends: traffic is 38% of weekday. A schedule keeps p95 under 1 s.
- 5 Risk
support-agent has no failover capacity in US-East
H100 availability is low at CoreWeave and none at AWS in this region. A node loss means degraded latency.
- 6 Risk
sql-copilot p95 crossed 1 s twice this week
Both spikes lined up with Lambda maintenance windows. Probes caught them 6 and 9 minutes before users reported.
View as table
| Day | Used | Idle |
|---|---|---|
| Sep 24 | $837 | $655 |
| Sep 25 | $990 | $561 |
| Sep 26 | $597 | $927 |
| Sep 27 | $585 | $924 |
| Sep 28 | $939 | $615 |
| Sep 29 | $872 | $629 |
| Sep 30 | $953 | $581 |
| Oct 1 | $876 | $645 |
| Oct 2 | $896 | $640 |
| Oct 3 | $589 | $987 |
| Oct 4 | $580 | $988 |
| Oct 5 | $863 | $637 |
| Oct 6 | $867 | $647 |
| Oct 7 | $940 | $617 |
| Oct 8 | $889 | $682 |
| Oct 9 | $933 | $611 |
| Oct 10 | $637 | $938 |
| Oct 11 | $629 | $943 |
| Oct 12 | $934 | $557 |
| Oct 13 | $872 | $647 |
| Oct 14 | $884 | $604 |
| Oct 15 | $883 | $674 |
| Oct 16 | $900 | $626 |
| Oct 17 | $610 | $964 |
| Oct 18 | $630 | $920 |
| Oct 19 | $891 | $628 |
| Oct 20 | $856 | $643 |
| Oct 21 | $960 | $556 |
| Oct 22 | $889 | $656 |
| Oct 23 | $875 | $616 |
Idle spend by endpoint
All endpoints as CSV- support-agent $9,6138× H100 · 41% utilized · $16,294 in 30 days
- transcribe $5,0332× H100 · 22% utilized · $6,453 in 30 days
- sql-copilot $3,0744× H200 · 68% utilized · $9,607 in 30 days
- doc-summarize $2,1944× H100 · 83% utilized · $12,906 in 30 days
- moderation $3111× A100 80GB · 57% utilized · $723 in 30 days
Fleet utilization by hour
Average of the last 4 weeks, UTC. Light cells are GPUs you paid for while they waited.