Total Cost / GPU / hr$1.257/hr
Capital / GPU / hr$0.802/hr63.8% of total
Operating / GPU / hr$0.455/hr36.2% of total
Total CapEx$18.0M$35K per GPU
TCO / PFLOP$0.635per hour
TCO / Training PFLOP$1.677per hour
Capital vs Operating Split
Cost composition for H100 SXM (InfiniBand)
Capital63.8%
Operating36.2%
Annual Operating Costs
$2,013,464 per year
Capital Cost Breakdown
Upfront investment — 8-GPU Server × 64 servers (16 racks)
| Component | Amount | % of Total |
|---|---|---|
| Servers | $12,800,000 | 71.1% |
| Network Fabric | $5,120,000 | 28.5% |
| Rack Infra (rack, PDUs, setup) | $70,400 | 0.4% |
| Total CapEx | $17,990,400 | 100.0% |
GPU Performance Context
H100 SXM specifications and efficiency metrics
FP8 TFLOPS1,979
Effective Training TFLOPS750
Inference (tok/s)90
Memory BW (TB/s)3.1
Memory (GB)80
Power (kW)11.0
GPUs per Server8
Utilization85.0%