fp8 vs fp4 vs bf16: what the precision field tells you and what it doesn't

The precision field next to an endpoint is the provider's declaration of the number format its copy of the model's weights runs at. bf16 is 16 bits per value, fp8 is 8, fp4 and int4 are 4. Fewer bits means less memory, faster serving and a lower price, and in principle less fidelity to the original weights. The field tells you the format the provider reports. It does not tell you what the endpoint returns, and many endpoints declare nothing at all.

The formats

• bf16 and fp16. Sixteen bits per value. The format most open-weight models are released in. Twice the memory of fp8.
• fp8. Eight bits. Supported natively on recent accelerators, so it is both smaller and faster. Some models are trained in fp8 and released that way, in which case fp8 is the original, not a reduction.
• fp4, mxfp4, int4, nvfp4. Four bits, usually with a shared scale per small block of values. Some models are released with their largest layers already in a four-bit block format. Done carefully on a large model, the loss on many tasks is small; done carelessly, it is not.
• Unknown. No declaration. Common. It means the provider has not said, not that the endpoint is full precision.

Why fewer bits is not automatically worse

Quantization is a conversion, and conversions vary. The calibration data, which layers are left at higher precision, whether activations and the key-value cache are reduced too, and which kernels run the result all matter more than the headline bit count. A careful four-bit endpoint can track the full-precision reference more closely than a careless eight-bit one. And when a model's maker released it at 8 or 4 bits, an endpoint declaring that format is serving the original.

So the declared precision is a format, not a quality. Two endpoints at the same declared precision can return different text, and an endpoint at a lower declared precision can return the same text as the reference.

What the field cannot show

• Which inference engine and kernels serve the request.
• Whether activations and the key-value cache are reduced, not only the weights.
• How the quantization was calibrated.
• The context length the endpoint actually honors, which is sometimes shorter than the model's.
• Anything that changed after the listing went up.

Each of these changes outputs. None of them appears in the response.

How to read it

Treat the field as a declaration and pair it with a measurement. Bitdepth's measurement is the same prompt set, every endpoint of the model, every night, scored against a reference built from the endpoints that declare full precision, with each endpoint's night-to-night disagreement with itself as the noise floor. The declared precision prints beside the measured grade so that the two can be compared, and a change in the declaration is recorded on the night it happens. How a grade is made. Why the same model answers differently on different providers.

Numbers against a stated reference. Not a claim about intent. Nothing on this site describes why any provider serves the way it does.