Best GPUs for Fine-Tuning

Fine-tuning covers a wide range of memory requirements, and the method you choose matters far more than the model size. Full fine-tuning updates every weight and therefore carries the same optimizer-state cost as pretraining — several times the parameter count in bytes. Parameter-efficient methods train a small number of added weights instead, cutting the requirement to roughly the size of the frozen model plus activations.

LoRA freezes the base model and trains low-rank adapter matrices, typically a fraction of a percent of the original parameters. QLoRA goes further by holding the frozen base in 4-bit precision, which is what makes single-GPU fine-tuning of large models practical. In exchange you accept some quality ceiling relative to a full fine-tune, which for most task-adaptation work is not the binding constraint.

That splits the hardware decision cleanly. A QLoRA run on a 7B–13B model fits on a single 24 GB card. A LoRA run on a 70B model wants 48–80 GB. A full fine-tune of anything above about 7B needs a multi-GPU node with sharding and behaves like a training job — see the LLM training page for how to size that.

Because fine-tuning jobs are short and bursty compared with pretraining, they are a good fit for on-demand and spot capacity. Spot pricing is meaningfully lower but instances can be reclaimed, so checkpoint frequently enough that a reclaim costs minutes rather than hours. Compare on-demand and spot rates in the table below.