Context is the active conversation window
Context length is the number of tokens a runtime can keep available while responding. A token is a model-specific text unit, not exactly a word or character.
A model publisher may document a maximum context length. That maximum describes the model configuration; it does not say that every computer can hold that much active context.
Choose an optional target in LLMGauge
The single-model calculator and recommendations workflow let you type a target context or choose one of four optional presets: 4,096, 8,192, 16,384, or 32,768 tokens. The field starts empty. No preset is a default, and LLMGauge does not turn a model's published default or maximum into a target for you.
In recommendations, the target is checked separately against each model's own documented context limit. A target above that maximum produces an unavailable estimate. A published maximum is not a promise that your hardware can run the model at that context.
What the KV-cache estimate means
While a model responds, the runtime keeps working information about earlier tokens. This is commonly called a key-value cache, or KV cache. A longer target can require more cache memory in addition to the model weights and runtime buffers.
LLMGauge shows an approximate FP16 cache estimate only when a model has supported, sourced architecture metadata. Supported text-only Llama catalog entries are Llama 3.1 8B Instruct, Llama 3.2 1B and 3B Instruct, and Llama 3.3 70B Instruct. Other supported entries are dense Qwen3 4B, 8B, 14B, and 32B, Qwen3-30B-A3B-Instruct-2507, and Qwen3-Coder-30B-A3B-Instruct. Cache estimates remain unavailable for other architectures.
The estimate uses 2 × transformer layers × KV heads × head dimension × 2 bytes × target context tokens, then divides by 2^30 to report GiB. The first 2 accounts for keys and values; 2 bytes is the FP16 assumption. The inputs come from sourced architecture metadata and the target you explicitly entered or selected.
The interface rounds the displayed size for readability. Fit assessment uses the unrounded estimate, so display rounding does not change the fit decision.
When the cache affects fit
A cache estimate affects memory fit only when the selected execution path identifies its placement. Explicit CPU execution counts it in system RAM. An explicit full-GPU choice with a CUDA or Vulkan backend and a discrete GPU counts it in dedicated VRAM. Apple unified-memory mode counts it in the single shared memory pool.
With automatic or partial-offload execution, or any otherwise unverified placement, LLMGauge still shows a supported estimate but excludes it from fit accounting. Placement remains advisory because the app cannot confirm how a particular runtime allocates memory.
Use the result as planning guidance
Start with a target that fits the work you actually do and leave memory headroom. The cache estimate is approximate, not a measurement of a running process.
A model's documented maximum is not a recommendation for your hardware. Runtime versions, backends, drivers, operating systems, other applications, and model-specific behavior can all affect whether a configuration runs. LLMGauge does not guarantee backend support or predict performance.