GPU Capacity Reservation: What It Is & When to Use It

GPU capacity reservation lets you lock in a specific amount of GPU compute (like a number of A100 or H100 chips) on a cloud provider for a fixed term, typically 1 to 3 years, in exchange for a lower hourly rate. You tell the provider how many GPUs you want, for how long, and they guarantee availability. The calculator below estimates the cost of a reservation versus on-demand pricing, helping you decide if committing makes sense for your AI startup.

Formula

The core math is simple: Monthly cost = GPUs × hourly rate × hours used. For on-demand, you pay the full hourly rate for every hour you run. For reserved, you pay a lower hourly rate, but you are billed for every hour in the month regardless of whether you use the GPUs.

• On-demand monthly cost = gpu × on_demand × hours
• Reserved monthly cost = gpu × reserved × hours
• Monthly savings = on-demand cost − reserved cost
• Total savings over term = monthly savings × months

This assumes you run the GPUs continuously for the entire month. If you shut them down, on-demand costs drop, but reserved costs do not—so the formula overestimates savings for intermittent workloads.

Worked Example

Using the defaults: 8 GPUs, on-demand rate $2.50/hour, reserved rate $1.50/hour, 730 hours per month, 12-month term.

1. On-demand monthly: 8 × 2.50 × 730 = $14,600
2. Reserved monthly: 8 × 1.50 × 730 = $8,760
3. Monthly savings: $14,600 − $8,760 = $5,840
4. Total savings over 12 months: $5,840 × 12 = $70,080

That is a 40% discount on the hourly rate, but it only works if you actually use the GPUs for all 730 hours. If you only train for 200 hours a month, on-demand would cost 8 × 2.50 × 200 = $4,000, while reserved still costs $8,760—a loss.

What GPU Capacity Reservation Actually Is

GPU capacity reservation is a contractual commitment. You reserve a specific quantity of a specific GPU type (e.g., 8× NVIDIA A100 40GB) in a specific region (e.g., us-east-1) for a term (1 or 3 years). In return, the cloud provider gives you a lower per-hour price compared to on-demand. The reservation is not a physical server you own; it is a billing and capacity guarantee. The provider ensures that when you launch instances, the reserved capacity is available, even during shortages.

This is distinct from spot instances (interruptible, steep discount, no guarantee) and on-demand (pay-as-you-go, full price, no commitment). Reservations sit in between: lower cost, but you pay whether you use it or not.

How It Works in Practice

Cloud providers (AWS, Google Cloud, Azure) offer reservation options. The mechanics are similar across providers:

• You choose the GPU type, quantity, region, and term length.
• You pay a lower hourly rate for the entire term, billed monthly (or upfront for additional discount).
• The reserved capacity is tied to that specific region and GPU type—you cannot move it to another region without paying conversion fees.
• If you stop using the GPUs, you still pay the reserved rate.
• Some providers offer “convertibleâ€� reservations that allow changing GPU type, but they cost more and have slightly higher rates.

For example, AWS calls this “Savings Plans� or “Reserved Instances�; Google calls it “Committed Use Discounts�; Azure calls it “Reserved VM Instances.� All follow the same principle.

When an AI Startup Should Use Reservations

Reservations make sense when you have predictable, continuous GPU demand. Common scenarios:

• You run a model training job that takes 3 months of 24/7 GPU usage.
• You host an inference API that serves traffic 24/7 and needs guaranteed uptime.
• You have a team that runs experiments constantly, with GPUs never idle.
• Your workload is stable enough that you can forecast usage for the next 12 months.

If your usage is spiky—like occasional fine-tuning or bursty inference—reservations may not pay off. You would be paying for idle capacity.

When to Avoid Reservations

Avoid reservations if any of these apply:

• You are still experimenting with model architectures and might switch GPU types.
• Your usage is intermittent (e.g., training only on weekends).
• You are a small startup with uncertain funding—you might not need GPUs in 6 months.
• You rely on spot instances for cost savings and can tolerate interruptions.

Also, beware of the region lock. If you reserve GPUs in us-east-1 but later need to move to eu-west-1 for data residency, you cannot transfer the reservation without penalties.

Common Mistakes and Misconceptions

A frequent mistake is assuming reservations are always cheaper. They are only cheaper if you use the GPUs enough. The break-even point is when your on-demand cost equals your reserved cost. If your utilization is below about 60% (depending on the discount), on-demand is cheaper.

Another misconception: “Reserved means I get a dedicated server.� No—you still share the physical host with other customers. The reservation only guarantees capacity, not exclusivity.

Finally, some startups reserve GPUs for a 1-year term but then realize they need a different GPU type (e.g., moving from A100 to H100). Convertible reservations help, but they have higher rates and still lock you to a family.

Costs and Alternatives

Reservations typically offer 30–60% savings over on-demand, but the exact discount varies by provider, GPU type, and term length. For a 3-year term, discounts are larger but the commitment is riskier.

Alternatives to reservations:

• Spot instances: Up to 90% discount, but can be terminated at any time. Best for fault-tolerant, non-critical workloads.
• On-demand: Full price, no commitment, maximum flexibility.
• On-premise hardware: High upfront cost, but no recurring cloud fees. Makes sense if you have sustained >80% utilization.
• Serverless GPU: Some providers offer per-second billing with auto-scaling, but at a higher rate than on-demand.

Frequently Asked Questions

How much can I save with a GPU reservation? Typically 30–60% off on-demand rates, depending on provider, term, and GPU type. Use the calculator above with your actual rates to estimate.

Can I cancel a reservation? No, reservations are non-refundable. You pay for the entire term even if you stop using the GPUs. Some providers allow selling unused reservations on a marketplace, but that is not guaranteed.

What happens if I need more GPUs than I reserved? You can launch additional on-demand instances, but they will be at on-demand prices. Reservations only cover the exact quantity you committed to.

Is a reservation the same as a dedicated host? No. A reservation guarantees capacity, but you still share the physical server with other tenants. Dedicated hosts are a separate, more expensive option.

How do I know if my startup should reserve GPUs? If your GPUs run more than 60–70% of the time for the next 12 months, reservations likely save money. If you are unsure, start with on-demand and track utilization for a month before committing.

Related

• How to Block Chrome: Stop It From Opening
• Solved Windows 10 Freezes Locks Up Randomly Easily?
• Galaxy Tab Pro S vs iPad Pro: Key Differences

in gpu | Number of GPUs | - | 8 | 1 | 1000
in on_demand | On-demand hourly rate per GPU | $ | 2.50 | 0 | 100
in reserved | Reserved hourly rate per GPU | $ | 1.50 | 0 | 100
in hours | Hours per month (avg) | h | 730 | 1 | 744
in months | Reservation term | months | 12 | 1 | 36
out on_cost | On-demand monthly cost | $ | gpu * on_demand * hours | 0
out res_cost | Reserved monthly cost | $ | gpu * reserved * hours | 0
out savings | Monthly savings | $ | on_cost - res_cost | 0
out total_savings | Total savings over term | $ | (on_cost - res_cost) * months | 0