Q4_K_M
The model-specific source lists this GGUF quantization; its memory value remains an approximate planning input, not a file size or runtime guarantee.
- Bits per weight
- 4.5
Model catalog
Llama 3.2 1B Instruct
Meta's compact instruction-tuned Llama model for lightweight local text generation.
Model overview
Available quantizations
Lower-bit quantizations generally use less memory. The compatibility calculator uses these candidates and its transparent approximate memory assumptions; it does not make a performance guarantee.
Q5_K_M
The model-specific source lists this GGUF quantization; its memory value remains an approximate planning input, not a file size or runtime guarantee.
- Bits per weight
- 5.5
Q8_0
The model-specific source lists this GGUF quantization; its memory value remains an approximate planning input, not a file size or runtime guarantee.
- Bits per weight
- 8
Compatibility notes
Estimates use the cataloged model-weight size (or a parameter-based estimate), apply weight overhead, and include standard runtime overhead. Context/KV-cache estimates are shown separately only when a target context and supported sourced architecture metadata are available. Real requirements vary with runtime, drivers, and other system use. A model page cannot determine compatibility without your hardware profile; use the calculator for that assessment.
KV-cache architecture metadata
Meta llama-models Llama 3.2 1B Instruct architecture configuration; verified value, last verified 2026-10-02. Meta lists 16 layers, model dimension 2048, 32 attention heads, and 8 KV heads for this exact text-only variant. Head dimension is derived as 2048 ÷ 32 = 64. The estimate assumes conventional full-context attention with FP16 cache elements.
View KV-cache architecture sourceMeta Llama 3.2 model card; verified value, last verified 2026-09-23. Parameter and context metadata come from the publisher's model card. GGUF quantization candidates are tracked separately and may be community-converted. View source