OpenRouter finds no clear quality gap between fp8, fp4 and fp16
OpenRouter found no detected accuracy difference between fp8, fp4 and fp16 in a test of Qwen3.8 27B, although the company cautions that the result covers one model and workload. Its pricing analysis found that lower precision was usually cheaper, but not in every case.
- OpenRouter detected no accuracy difference among fp8, fp4 and fp16 across 240 scored tasks.
- The test covered Qwen3.8 27B across six provider endpoints.
- fp8 was cheaper than 16-bit for 14 of 24 models tested.
- fp4 was cheaper than fp8 for 26 of 33 models tested.
- Users can pin accepted precision with provider.quantizations.
Quantization trades memory for lower precision
LLM quantization stores a model's weights in fewer bits than the format used during training. According to OpenRouter, most current models use 16-bit FP16 or BF16, while fp8 uses 8 bits per weight and fp4 or int4 uses 4 bits.
For a 27B model, the weights require 54 GB in bf16 or fp16, 27 GB in fp8 and 13.5 GB in fp4, according to the company's table. These figures cover weights only. A running model also needs memory for its KV cache, which stores attention state for each token in the context.
Using fewer bits allows the same GPU to hold a larger model or serve more requests. The tradeoff is that each weight is rounded to a value the smaller format can store, which can reduce accuracy. OpenRouter says that post-training quantization can be dynamic or static, while quantization-aware training simulates this rounding during training.
Qwen3.8 27B results showed overlapping scores
OpenRouter tested 280 exact-answer tasks through six endpoints serving Qwen3.8 27B on October 20, 2026. Cerebras served the model at fp16, Parasail, CoreWeave and AkashML at fp8, and Darkbloom and ModelRun at fp4.
The test included multi-step arithmetic, code tracing, sorting, long-context recall and a four-step lookup chain. Across 240 scored tasks, the six endpoints scored between 98.3% and 99.2%. The fp16 Cerebras endpoint and the fp8 endpoints scored between 98.3% and 98.8%, while the fp4 endpoints scored 98.3% and 99.2%.
OpenRouter says the 95% confidence intervals on the totals overlapped, and no task family separated the precisions. In a separate 40-task multiplication test, the Darkbloom fp4 endpoint scored 26 correct answers, compared with 30 to 33 for the other endpoints. The company said the difference was too small to identify as a precision effect, with an estimated chance of about 7.5% that a gap of that size could arise without a real difference.
OpenRouter stresses that the test does not show 4-bit quantization is safe for every model or workload. It recommends testing individual prompts before moving traffic to a 4-bit endpoint.
Lower precision usually costs less, with exceptions
OpenRouter examined 1,455 provider endpoints across 470 models. Among all endpoints, 865 did not report a precision, 343 were labeled fp8, 123 fp4, 70 bf16, 20 int4, 13 nvfp4, 12 mxfp4, eight fp16 and one fp32. The company said fp8 was the most common labeled precision.
In a comparison of the cheapest endpoint at each precision, 8-bit was cheaper than 16-bit for 14 of 24 models and more expensive for seven. Four-bit was cheaper than 8-bit for 26 of 33 models and more expensive for four. The median price ratio was 0.94 for 8-bit versus 16-bit and 0.76 for 4-bit versus 8-bit.
Examples show why users cannot assume the lowest precision always has the lowest price. For Qwen3.8 27B, the listed blended price was $0.581 per million tokens at 16-bit, $0.342 at 8-bit and $0.140 at 4-bit. For gpt-oss-120b, 8-bit cost $0.100 compared with $0.067 at 16-bit. MiniMax M3 cost $0.412 at 4-bit and $0.388 at 8-bit.
For API users, OpenRouter says provider.quantizations pins the precision they receive. Listing every acceptable precision gives the router more than one endpoint to use as a fallback.
OpenRouter: LLM Quantization: What fp8 and fp4 Do to Quality and Price
