Q4_K_M
A commonly available lower-memory GGUF variant; the listed memory input is an estimate, not the repository file size.
- Bits per weight
- 4.5
Model catalog
Mistral Small 3.2 24B Instruct
Mistral's 24B instruction-tuned model; the memory estimate covers text weights and not its image-processing path.
Model overview
Available quantizations
Lower-bit quantizations generally use less memory. The compatibility calculator uses these candidates and its transparent approximate memory assumptions; it does not make a performance guarantee.
Q8_0
A higher-memory GGUF variant where listed by the conversion repository; the listed memory input remains approximate.
- Bits per weight
- 8
Compatibility notes
Weight estimates are approximate and include a runtime overhead. Real requirements vary with context length, runtime, drivers, and other system use. A model page cannot determine compatibility without your hardware profile; use the calculator for that assessment.
Mistral Small 3.2 model documentation; verified value, last verified 2026-09-24. Mistral's documentation lists a 128K context and Apache-2.0 license and marks Small 3.2 deprecated for new integrations; the publisher model card identifies the 24B instruct variant. This entry is for local GGUF memory planning, not an API recommendation. Image-processing memory is outside this text-weight estimate. The listed GGUF candidates are a separate bartowski community conversion. View source