What on-demand GPU instances are
An on-demand GPU instance is a virtual machine with a graphics processing unit (GPU) attached, which suits the parallel computation behind machine learning. You start it when you need it and stop it when you finish, with no long-term agreement in between. The provider sells whatever capacity it has available to anyone who asks, so you can change size or region between runs. Billing models differ across providers, so read how each one bills before you begin.
What reserved GPU instances are
A reserved GPU instance is capacity you agree to hold for a fixed term. The commitment stands whether or not your jobs use that capacity on a given day. A reservation can also keep a machine ready for you and protect your access when demand for a model rises, though whether it does depends on the provider. Terms and length vary, so read them before you agree to anything.
Utilisation settles the choice
The deciding factor is utilisation, meaning the share of time a machine does useful work. A busy GPU makes a commitment useful, since it keeps you on hardware you need anyway. A GPU that sits idle between bursts means the commitment covers time you never use.
Flexibility is the other side of that trade. On-demand keeps every choice open, while reserved fixes the choice in advance. Neither is better in general, only better for a given pattern of use. The GPU models on offer matter too, because the model that suits a job may not be the one with the most spare capacity.
When on-demand wins
On-demand fits work where the plan is still changing. It also fits jobs that run now and then, and it avoids paying for capacity while nothing runs. Keeping the option open matters more when the load is hard to predict.
- Experiments, where the right size or model may change from week to week.
- Jobs that arrive in bursts, with long quiet stretches between them.
- Testing a model before deciding whether it deserves a longer commitment.
- A small team without a steady forecast of demand.
When reserved wins
Reserved capacity suits steady work that runs most days on the same kind of machine. Long training projects and services that must answer users at any moment both fit. Predictable load is the real signal here, more than the size of the project.
- A model serving live traffic that stays busy day and night.
- A training run that holds the same machine for weeks at a time.
- A capacity forecast stable enough to commit to with confidence.
- A team that plans well ahead of launches and knows its peak needs.
Spot capacity as another option
Spot, or interruptible, capacity is spare GPU time sold for less than on-demand. Providers can reclaim it at short notice, so the trade suits some work and rules out other work. The lower price is not the only question. Think about what a sudden stop would mean for your results before you choose it.
Spot suits work that can stop and restart safely, such as batch jobs that save progress to storage as they run. Live services that must answer every request do not fit. Check how much warning the provider gives before reclaiming a machine, because that varies by provider.
Which to pick
Match the option to how your work behaves, since that matters more than any provider's name.
- A researcher or hobbyist running experiments should start on-demand and move to a commitment only after the workload settles.
- A small team with uneven traffic should start on-demand and watch utilisation for a while before committing.
- A company serving a model that never goes quiet should reserve the steady base and use on-demand for peaks.
- A team with interruptible batch work may find spot capacity suits those jobs, provided they can restart cleanly.
To narrow the field, read the guide to choosing a GPU and compare options on the GPU board. Terms, notice periods and rules differ by provider, so check each provider's own terms before you commit.