GGUF

GGUF is one of the file formats used to package a model's weights (its "brain file") into a single file, and it has become the standard packaging format for the "local model" ecosystem — running AI on your own PC. It originated from llama.cpp, the pioneering project for local model execution.

The key is its pairing with quantization — compression that lowers numerical precision to shrink a model's size. Because of this, the same model gets released in multiple GGUF versions, like Q4 or Q8. Smaller numbers mean a lighter file but a slightly less capable model, so "picking the version that fits your GPU memory" is the first hurdle every local user faces.

When a new model is released, it's customary for the community to produce GGUF versions within days. "When's the GGUF coming?" is practically the standard first comment under any new model announcement — so whenever you see this term, read it as a sign that the conversation is about people trying to run the model on their home computers.

"Community GGUF conversion rush right after new model release" — articles about open-weight models are almost always followed by discussion of local execution, and this file format sits at the center of it.