Runtime basics

GPU Offloading: Full, Partial, and CPU-Only

See what it means to run a local model entirely on a GPU, partly on a GPU, or on the CPU.

Last reviewed 2026-09-22.

Full GPU execution

Full GPU execution means the runtime can place the model's relevant layers and buffers on a supported GPU within the available memory. This is often the simplest way to use a discrete GPU, but fitting the weights does not guarantee a particular speed.

The runtime still needs a compatible backend and driver. A GPU listed in a hardware catalog is not the same thing as a tested configuration for every operating system or llama.cpp build.

Partial offload

Partial offload keeps some layers or buffers on the GPU and leaves the rest in system memory. It can make a model usable when the entire model does not fit in dedicated VRAM, but data has to move between memory pools and the result can be more sensitive to configuration.

Because the split depends on the model, context, backend, and runtime settings, LLMGauge reports partial offload as a compatibility category rather than predicting a speed improvement.

CPU-only execution and integrated graphics

CPU-only execution means the model is planned for system RAM without relying on dedicated GPU memory. It can be the appropriate fallback when there is no discrete GPU or when GPU memory is insufficient.

Integrated GPUs can share system memory, but the amount and behavior are platform-dependent. LLMGauge treats integrated/shared-memory graphics conservatively and does not count shared RAM as dedicated VRAM in its first-pass classification.

Continue with LLMGauge

References

These references support the concepts explained here. LLMGauge simplifies them for planning and does not present the guides as runtime documentation or performance benchmarks.